REVIEW 5 major objections 6 minor 26 references
A Performance Analysis of You Only Look Once Models for Deployment on Constrained Computational Edge Devices in Drone Applications
T0 review · 5 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read Quantizing the smallest YOLOv8 model to 8-bit integers lets it run at about 65 FPS on the Jetson Orin NX, clearing the real-time bar for drone detection, while the Raspberry Pi 5 stays at 7-8 FPS and misses it.
desk verdict Useful but under-specified edge benchmark; the performance ordering is plausible, but the INT8 accuracy claims need calibration details before I'd trust them. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument rests on a three-way benchmark matrix that pairs each YOLOv8 variant (nano and small) with three numerical precisions (FP32, FP16, INT8) on three edge devices, using TensorRT for the Jetson modules and the NCNN framework for the Raspberry Pi 5. The load-bearing measurements are the mean inference speed (frames per second), inferences per minute, and energy per inference, with detection quality tracked by mAP50 and mAP50-95 on a fixed 640x640 input. The decisive mechanism is INT8 post-training quantization: it reduces the bit width of weights and activations to 8-bit integers, cutting compute and memory enough to push the nano model past 60 FPS on the Orin NX while keeping the accuracy loss within what the authors call acceptable for real-time use. The cloud comparison is carried by round-trip time, which separates model processing latency from communication latency.
What would settle it
Re-run YOLOv8n INT8 on the same Orin NX using a calibration set that is representative of the deployment imagery (for example, a held-out subset of the same 6000 images or an independent outdoor drone dataset) and compare mAP50-95 against the reported 0.719; if the value shifts by several points or the FPS changes materially, the paper's central trade-off is calibration-dependent.
Extended reading notes
Core claim
Using a 6000-image dataset of ground-robot targets captured from drones in an indoor testbed, the paper measures mean inference speed, throughput, energy per inference, and detection accuracy for YOLOv8n and YOLOv8s across the Jetson Orin Nano, Jetson Orin NX, and Raspberry Pi 5. Its central claim is that YOLOv8n at INT8 precision deployed through the TensorRT pipeline on the Orin NX is the best tested configuration, delivering 65.83 FPS with mAP50-95 of 0.719; this is roughly a 26 percent speed gain over the same model at FP32 (52.19 FPS) at the cost of about 12.5 mAP points. The Raspberry Pi 5, running FP32 via the NCNN framework, reaches only 8.47 FPS for YOLOv8n and 7.32 FPS for YOLOv8s, which the authors judge insufficient for real-time drone control. In the end-to-end testbed, the Orin NX achieves a round-trip time around 30 ms, whereas a cloud GPU instance with much faster model processing (6.82 ms) has a total round trip near 348 ms because of communication latency, so edge deployment wins on end-to-end latency. From these results the paper derives a configuration guide linking operational needs such as long-duration surveillance, critical tracking, and high-precision inspection to specific device-and-quantization choices.
Load-bearing premise
The load-bearing premise is that the INT8 accuracy loss was measured honestly: the paper does not describe which calibration dataset, sample count, or algorithm the quantization step used, and the reported mAP drop and the acceptability verdict both depend on that step.
Editorial extensions
If this is right
- A Jetson-class edge GPU running YOLOv8n in INT8 is sufficient for real-time drone object detection, with throughput around 66 FPS and a round-trip time near 30 ms in the testbed.
- INT8 quantization is the fastest precision on both Jetson devices for both model sizes, improving FPS by roughly a quarter on the Orin NX while cutting energy per inference.
- The Raspberry Pi 5, despite its low power draw, is not a viable platform for real-time detection in this pipeline; its role is limited to low-demand, non-real-time processing.
- Cloud inference cannot meet real-time drone requirements from a remote region because communication latency (about 341 ms) dominates the roughly 7 ms of model processing time.
- FP16 offers an intermediate trade-off: near-lossless accuracy with most of INT8's speed gain, recommended when precision matters more than maximum throughput.
Reading between the lines
- Because the benchmark uses a single-class indoor dataset, the exact FPS and mAP numbers are dataset-specific, but the qualitative ordering—Jetson INT8 beats FP16 beats FP32, and either Jetson beats the Raspberry Pi—should transfer to other deployment workloads since it follows from hardware throughput.
- The unstated INT8 calibration step is the biggest reproducibility risk; re-running the quantization with a calibration set drawn from the deployment imagery rather than from the training set could change both the 0.719 mAP50-95 figure and the recommendation that INT8 accuracy is acceptable.
- A hybrid policy that runs the cheap nano model on the edge and forwards only low-confidence frames to the cloud could combine the 6.82 ms cloud processing time with edge-level end-to-end latency, an option the paper mentions only as a future direction.
- Measuring energy at a fixed 640x640 input and default clocks leaves headroom: enabling dynamic voltage and frequency scaling or testing small batches would likely shift the energy-per-inference ranking between the Orin Nano and the Orin NX.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports a performance evaluation of YOLOv8n and YOLOv8s object detectors on three edge devices (Jetson Orin Nano, Jetson Orin NX, and Raspberry Pi 5) across FP32, FP16, and INT8 precision levels, and compares edge processing with an AWS SageMaker cloud endpoint within a drone video pipeline. It finds that YOLOv8n with INT8 quantization on the Orin NX is the fastest configuration (about 66 FPS), that the Raspberry Pi 5 cannot meet real-time requirements (7-8 FPS), and that edge processing achieves much lower round-trip latency than cloud processing (35 ms versus 348 ms). The paper also reports energy per inference and provides configuration recommendations for different drone operation scenarios.
Significance. If the measurements are correct, the paper provides a useful, directly applicable benchmark for practitioners selecting hardware and quantization schemes for drone-based object detection. Its strengths are the realistic testbed, the use of standard metrics (mAP, FPS, EPI, RTT), and the explicit edge-versus-cloud latency comparison, which is often missing from isolated benchmarks. The central qualitative ordering (Orin NX fastest, INT8 fastest on Jetson devices, RPI5 not real-time, edge beats cloud on RTT) is plausible and aligns with prior results on Jetson platforms. The contribution is incremental but potentially valuable as applied engineering data; its value depends on the reproducibility of the reported numbers, which is currently weakened by missing calibration details, absent dispersion measures, and several internal inconsistencies that need to be resolved.
major comments (5)
- [Section 3.1.3 / Table 4] The INT8 post-training quantization procedure is not described: the paper never states which calibration dataset was used, how many samples it contained, or which calibration algorithm TensorRT employed, and Sections 3.1.1-3.1.2 do not specify the train/validation/test split used to compute the mAP values in Table 4. Because the central recommendation of YOLOv8n INT8 on the Orin NX rests on judging the 12-point mAP50-95 drop from 0.842 to 0.719 acceptable, the accuracy trade-off is not independently checkable as reported.
- [Section 4.1 / Table 4] The mean iteration times reported in Section 4.1 are not consistent with the MIS (FPS) values in Table 4, given the paper's own definition MIS = 1/mean inference time. For example, Orin Nano YOLOv8s FP32 is stated to take 42.97 ms (about 23.3 FPS) but Table 4 reports 27.00 FPS (37.04 ms), and Orin Nano YOLOv8n INT8 is stated at 23.16 ms (about 43.2 FPS) versus 44.9 FPS (22.27 ms). These discrepancies affect the reported speedups and need to be reconciled.
- [Section 5.1 / Table 5 vs Table 6] The same configuration (Orin Nano, YOLOv8s FP16) is reported with an edge RTT of 49.44 ms and processing time 28.90 ms in Table 5, but in Table 6 the edge RTT is 35.09 ms and the model processing latency is 32.59 ms. Since Section 5.2 uses the Table 6 numbers to argue that edge (35 ms) beats cloud (348 ms), the discrepancy must be explained and the two tables must be reconciled.
- [Table 4 / Section 4.3] The column labeled 'Energy Consumption (W·s)' in Table 4 actually reports mean power in watts: the values are 5.4-14.2 W, and the EPI formula in Table 3 uses mean power times 60 s divided by IPM. Labeling this column as energy in W·s misstates the measured quantity and makes the energy analysis confusing; the header and the discussion in Section 4.3 should be corrected (for example, to 'Mean Power (W)') and the measurement time window should be clarified.
- [Sections 4.1-4.3] No error bars, standard deviations, or confidence intervals are provided for the isolated FPS, mAP, and energy measurements, despite the averaging described in Section 3.2. Some comparative claims in Section 4.2, such as Orin Nano YOLOv8s FP16 (37.90 FPS) slightly exceeding Orin NX YOLOv8s FP32 (35.65 FPS), rely on small differences that may be within run-to-run variation, so the paper should report dispersion measures.
minor comments (6)
- [Section 6] The statement in Section 6 that INT8 is the fastest configuration 'across all evaluated devices' is an overstatement, because the Raspberry Pi 5 was only tested in FP32.
- [Section 7] Section 7 refers to 'recommendations (Table 6)', but the deployment recommendation table is actually Table 7.
- [Table 7] Table 7 labels mAP50-95 values as mAP50 in the 'Key Supporting Metrics' column; for instance, the Orin Nano YOLOv8s FP16 row cites mAP50: 0.8622, which is the mAP50-95 value from Table 4, while the actual mAP50 is 0.9771.
- [References] Reference [2] appears to be a placeholder with generic authors and a non-resolvable DOI, and the in-text attribution to 'Park et al. [25]' in Section 2.2 does not match reference [25], which is a paper on STT-MRAM power-aware quantization; these citations should be verified and corrected.
- [Figures 4 and 5] Figures 4 and 5 present the same data in different units but without consistent values, and their y-axis labels and the relationship to Table 4 should be clarified.
- [Table 5] Table 5's throughput column mixes formatting (for example, '28.5' versus '28.50') and lacks a units note, and the standard deviation columns are not explicitly defined in the text.
Circularity Check
No circularity: this is a direct measurement study; FPS, mAP, and energy figures are observed results, not derivations from their own inputs.
full rationale
The paper is an empirical benchmarking study. YOLOv8n and YOLOv8s are trained on a newly created dataset, then deployed on three edge devices under FP32, FP16, and INT8 configurations. The headline results — e.g., 52 FPS for YOLOv8n on the Jetson Orin NX and 65.83 FPS with INT8, with mAP50-95 values in Table 4 — are direct measurements. There is no fitted parameter that is later renamed as a prediction, no quantity that is defined in terms of another quantity that it is then claimed to predict, and no derivation chain that reduces to its own inputs by construction. The only self-citation is reference [26], used to describe the indoor drone testbed and the source of the dataset. That citation supplies infrastructure and experimental context, not a mathematical theorem or an assumed result that forces the FPS, mAP, or energy conclusions; the numerical outcomes are measured on that infrastructure and are externally checkable. The paper's limitations are real but do not amount to circularity: the TensorRT INT8 calibration procedure and the train/validation/test split are not specified, which makes the reported INT8 accuracy degradation difficult to reproduce, and Tables 5 and 6 contain an internal inconsistency for the Orin Nano YOLOv8s FP16 RTT (49.44 ms vs 35.09 ms). These are reproducibility and consistency concerns, not instances of a prediction being equivalent to an input by definition. Accordingly, no circular step is identified.
Assumptions & free parameters
assumptions (4)
- domain assumption The Ultralytics YOLOv8 models (v8.0.0) provide a valid and typical baseline for the YOLOv8n/s evaluation.
- domain assumption tegrastats and vcgencmd pmic_read_adc reports accurately reflect inference energy consumption.
- domain assumption The manually annotated 6000-image indoor dataset is correct and sufficiently representative for the accuracy comparisons.
- domain assumption TensorRT INT8 post-training quantization was calibrated on a representative dataset.
Cite this review
Pith. "Pith review of A Performance Analysis of You Only Look Once Models for Deployment on Constrained Computational Edge Devices in Drone Applications." pith.science (2026). https://pith.science/paper/X3RTCL5F
@misc{pith2026250215737,
author = {Pith},
title = {Pith review of: A Performance Analysis of You Only Look Once Models for Deployment on Constrained Computational Edge Devices in Drone Applications},
year = {2026},
howpublished = {\url{https://pith.science/paper/X3RTCL5F}},
note = {Machine review of arXiv:2502.15737}
}
read the original abstract
Advancements in embedded systems and Artificial Intelligence (AI) have enhanced the capabilities of Unmanned Aircraft Vehicles (UAVs) in computer vision. However, the integration of AI techniques o-nboard drones is constrained by their processing capabilities. In this sense, this study evaluates the deployment of object detection models (YOLOv8n and YOLOv8s) on both resource-constrained edge devices and cloud environments. The objective is to carry out a comparative performance analysis using a representative real-time UAV image processing pipeline. Specifically, the NVIDIA Jetson Orin Nano, Orin NX, and Raspberry Pi 5 (RPI5) devices have been tested to measure their detection accuracy, inference speed, and energy consumption, and the effects of post-training quantization (PTQ). The results show that YOLOv8n surpasses YOLOv8s in its inference speed, achieving 52 FPS on the Jetson Orin NX and 65 fps with INT8 quantization. Conversely, the RPI5 failed to satisfy the real-time processing needs in spite of its suitability for low-energy consumption applications. An analysis of both the cloud-based and edge-based end-to-end processing times showed that increased communication latencies hindered real-time applications, revealing trade-offs between edge (low latency) and cloud processing (quick processing). Overall, these findings contribute to providing recommendations and optimization strategies for the deployment of AI models on UAVs.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[25]
Power-aware Quantization Circuits in Analog In-Memory Computing with STT-MRAM Macro
Zhou, M.; Guo, Y.; Fu, J.; Cai, H. Power-aware Quantization Circuits in Analog In-Memory Computing with STT-MRAM Macro. In Proceedings of the 2023 IEEE International Magnetic Conference—Short Papers (INTERMAG Short Papers), Sendai, Japan, 15–19 May 2023; pp. 1–2. https://doi.org/10.1109/INTERMAGShortPapers58606.2023.10228662
-
[1]
Counting Vehicles with Deep Learning in Onboard UAV Imagery
Amato, G.; Ciampi, L.; Falchi, F.; Gennaro, C. Counting Vehicles with Deep Learning in Onboard UAV Imagery. In Proceedings of the 2019 IEEE Symposium on Computers and Communications (ISCC), Barcelona, Spain, 29 June–3 July 2019; pp. 1–6. https://doi.org/10.1109/ISCC47284.2019.8969620
-
[2]
Autonomous Navigation in Drones Using Deep Learning
Brown, A.; White, B. Autonomous Navigation in Drones Using Deep Learning. In Proceedings of the International Conference on Autonomous Systems, Barcelona, Spain, 13–17 March 2023; pp. 456–467. https://doi.org/10.5678/icas.2023.67890
-
[3]
Learning Vision-Based Flight in Drone Swarms by Imitation
Schilling, F.; Lecoeur, J.; Schiano, F.; Floreano, D. Learning Vision-Based Flight in Drone Swarms by Imitation. IEEE Robot. Autom. Lett. 2019, 4, 4523–4530. https://doi.org/10.1109/LRA.2019.2935377
-
[4]
Profiling Energy Consumption of Deep Neural Networks on NVIDIA Jetson Nano
Holly, S.; Wendt, A.; Lechner, M. Profiling Energy Consumption of Deep Neural Networks on NVIDIA Jetson Nano. In Proceedings of the 2020 11th International Green and Sustainable Computing Workshops (IGSC), Pullman, WA, USA, 19-22 October 2020; pp. 1–6. https://doi.org/10.1109/IGSC51522.2020.9290876
-
[5]
A Self-Learning Strategy for Task Offloading in UAV Networks.IEEE Trans
Sacco, A.; Esposito, F.; Marchetto, G.; Montuschi, P . A Self-Learning Strategy for Task Offloading in UAV Networks.IEEE Trans. Veh. Technol. 2022, 71, 4301–4311. https://doi.org/10.1109/TVT.2022.3144654
-
[6]
Real-Time Flying Object Detection with YOLOv8
Reis, D.; Kupec, J.; Hong, J.; Daoudi, A. Real-Time Flying Object Detection with YOLOv8. arXiv 2024, arXiv:2305.09972
arXiv 2024
-
[7]
Ultralytics YOLO, Version 8.0.0
Jocher, G.; Qiu, J.; Chaurasia, A. Ultralytics YOLO, Version 8.0.0. Online Resource, 2023. Available online: https://github.com/ ultralytics/ultralytics (accessed on 4 February 2025)
work page 2023
Show all 26 references
-
[8]
A Survey on the Convergence of Edge Computing and AI for UAVs: Opportunities and Challenges
McEnroe, P .; Wang, S.; Liyanage, M. A Survey on the Convergence of Edge Computing and AI for UAVs: Opportunities and Challenges. IEEE Internet Things J. 2022, 9, 15435–15459. https://doi.org/10.1109/JIOT.2022.3176400
2022
-
[9]
DeepEdgeBench: Benchmarking Deep Neural Networks on Edge Devices
Baller, S.P .; Jindal, A.; Chadha, M.; Gerndt, M. DeepEdgeBench: Benchmarking Deep Neural Networks on Edge Devices. arXiv 2021, arXiv:2108.09457
2021 arXiv
-
[10]
An Evolutionary Algorithm to Optimise a Distributed UAV Swarm Formation System
Stolfi, D.H.; Danoy, G. An Evolutionary Algorithm to Optimise a Distributed UAV Swarm Formation System. Appl. Sci. 2022, 12, 10218. https://doi.org/10.3390/app122010218
2022 doi
-
[11]
ABM-SpConv-SIMD: Accelerating Convolutional Neural Network Inference for Industrial IoT Applications on Edge Devices
Li, X.; Gong, X.; Wang, D.; Zhang, J.; Baker, T.; Zhou, J.; Lu, T. ABM-SpConv-SIMD: Accelerating Convolutional Neural Network Inference for Industrial IoT Applications on Edge Devices. IEEE Trans. Netw. Sci. Eng. 2023, 10, 3071–3085. https://doi.org/10.1109/TNSE.2022.3154412
2023
-
[12]
Benchmarking Convolutional Neural Network Inference on Low-Power Edge Devices
Ferraz, O.; Araujo, H.; Silva, V .; Fernandes, G.F.P . Benchmarking Convolutional Neural Network Inference on Low-Power Edge Devices. In Proceedings of the ICASSP 2023—2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece...
2023
-
[13]
DistrEdge: Speeding up Convolutional Neural Network Inference on Distributed Edge Devices
Hou, X.; Guan, Y.; Han, T.; Zhang, N. DistrEdge: Speeding up Convolutional Neural Network Inference on Distributed Edge Devices. In Proceedings of the 2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS), Lyon, France, 30 May–3 June 2022; pp. 1097–1107...
2022
-
[14]
Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous Vehicles
Liang, S.; Wu, H.; Zhen, L.; Hua, Q.; Garg, S.; Kaddoum, G.; Hassan, M.; Yu, K. Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous Vehicles. IEEE Trans. Intell. Transp. Syst. 2022, 23, 25345–25360. https://doi.org/10.1109/TIT...
2022
-
[15]
A Deep Learning Framework Performance Evaluation to Use YOLO in Nvidia Jetson Platform
Shin, D.J.; Kim, J.J. A Deep Learning Framework Performance Evaluation to Use YOLO in Nvidia Jetson Platform. Appl. Sci. 2022, 12, 3734. https://doi.org/10.3390/app12083734
2022 doi
-
[16]
GCGE-YOLO: Improved YOLOv5s Algorithm for Object Detection in UAV Images
Xiong, G.; Qi, J.; Wang, M.; Wu, C.; Sun, H. GCGE-YOLO: Improved YOLOv5s Algorithm for Object Detection in UAV Images. In Proceedings of the 2023 42nd Chinese Control Conference (CCC), Tianjin, China, 24–26 July 2023; pp. 7723–7728. https://doi.org/10.23919/CCC58697.2023.10240567
2023
-
[17]
YOLO-Tiny-attention: An Improved Algorithm for Fault Detection of Wind Turbine Blade
Hu, Y.; Wang, L.; Kou, T.; Zhang, M. YOLO-Tiny-attention: An Improved Algorithm for Fault Detection of Wind Turbine Blade. In Proceedings of the 2023 8th International Conference on Intelligent Computing and Signal Processing (ICSP), Xi’an, China, 21–23 April 2023; pp. 1228–12...
2023
-
[18]
5G for drone networking
Xu, C. 5G for drone networking. Trans. Emerg. Telecommun. Technol. 2022, 33, e4668. https://doi.org/10.1002/ett.4668
2022 doi
-
[19]
Photonics-Assisted Millimeter-Wave Communication System Based on Low-Bit Gaussian Mixture Model Adaptive Vector Quantization
Cai, Y.; Yue, L.; Zhu, M.; Lei, M.; Zhang, J.; Hua, B.; Luo, W.; Zou, Y.; Tian, L.; Ma, L.; et al. Photonics-Assisted Millimeter-Wave Communication System Based on Low-Bit Gaussian Mixture Model Adaptive Vector Quantization. IEEE Photonics J. 2022, 14, 1–9. https://doi.org/10....
2022
-
[20]
Placement and Allocation of Communications Resources in Slicing-aware Flying Networks
Coelho, A.; Fontes, H.; Campos, R.; Ricardo, M. Placement and Allocation of Communications Resources in Slicing-aware Flying Networks. arXiv 2021, arXiv:2112.07048
2021 arXiv
-
[21]
Post Training Quantization after Neural Network
Jiang, H.; Li, Q.; Li, Y. Post Training Quantization after Neural Network. In Proceedings of the 2022 14th International Conference on Computer Research and Development (ICCRD), Shenzhen, China, 7–9 January 2022; pp. 1–6. https://doi.org/10.1109/ ICCRD54409.2022.9730411
2022
-
[22]
Power-of-Two Quantization for Low Bitwidth and Hardware Compliant Neural Networks
Przewlocka-Rus, D.; Sarwar, S.S.; Sumbul, H.; Li, Y.; Salvo, B.D. Power-of-Two Quantization for Low Bitwidth and Hardware Compliant Neural Networks. arXiv 2022, arXiv:2203.05025. https://doi.org/10.48550/arXiv.2203.05025. Electronics 2025, 1, 0 25 of 25
-
[23]
Unified Scaling-Based Pure-Integer Quantization for Low-Power Accelerator of Complex CNNs
Al-Hamid, A.A.; Kim, H. Unified Scaling-Based Pure-Integer Quantization for Low-Power Accelerator of Complex CNNs. Electronics 2023, 12, 2660. https://doi.org/10.3390/electronics12122660
2023 doi
- [24]
-
[26]
A Digital Twin Concept to Prototype Multi-Drone Based Applications
Carramiñana, D.; Braga, L.R.; Bernardos, A.M.; Dobrzycki, A.D.; Bergesio, L.; Besada, J.A.; Casar, J.R. A Digital Twin Concept to Prototype Multi-Drone Based Applications. Presented at the 14th International Conference on the Internet of Things (IoT 2024), Oulu, Finland, 19–22...
2024
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.