{"id":"6f88c98e-1179-4893-9193-281dc32e14c9","arxiv_id":"2607.09835","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.5,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"YOLO26 matches prior YOLO accuracy on full data for fish mortality detection but needs more images than YOLOv8 and is fastest only in nano/small tiers on Raspberry Pi 5 CPU.","lead":"This benchmark compares YOLO26 to three earlier Ultralytics YOLO models for detecting dead fish in recirculating aquaculture tanks on both GPU and a Raspberry Pi 5. Newer architecture does not automatically win: accuracy is nearly identical with enough data, while data efficiency and CPU speed trade off by model family and size.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified","rationale":"The strongest claim is an empirical observation drawn directly from Table 3 (mAP50 range 93.77–94.81 %) plus the learning-curve and dual-hardware results (Figs. 4–5, Tables 4–5). That observation is sufficient to underwrite the practical recommendation that data availability, target hardware, and speed requirements must be considered jointly. The single-tank / cull-augmented / adult-salmon limitation is real for external generalizability, yet the paper never claims universal superiority rankings; it claims only that, under the conditions tested, architecture generation is not decisive for accuracy and that deployment metrics matter. Because that narrower claim is internally consistent and transparently caveated, no adjustment to the reader's CONDITIONAL verdict is warranted.","tokens_in":19249,"tokens_out":392,"duration_ms":4223,"concrete_test":"Re-train the twelve models on a second independent RAS mortality set (different species or juvenile stage, if available) using the same protocol; if the mAP50 spread remains ≤2 points and the relative data-efficiency / Pi-5 FPS orderings are preserved, the joint-selection claim is reinforced.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is modest and well-supported by the tabulated results: with full data, mAP50 varies by only 1.04 points across twelve Ultralytics models, while data-efficiency and CPU/GPU rankings diverge, so architecture alone is insufficient for edge selection. The reader's weakest assumption (representativeness of the single-tank, adult-salmon, cull-augmented mortality set) is already flagged by the authors in §4 and does not undercut the internal comparative rankings or the joint-selection recommendation. No hidden inconsistency, circular derivation, or unacknowledged confound appears load-bearing for the claim as stated.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"This paper benchmarks the new Ultralytics YOLO26 (NMS-free, edge-oriented) against YOLOv5u, YOLOv8 and YOLO11 at nano/small/medium scales on a binary fish-mortality detection task in a semi-commercial RAS. Twelve models are compared on full-dataset accuracy (mAP50/mAP50-95/precision/recall), data efficiency via learning curves over seven stratified subset sizes, GPU (A100) forward-pass/post-process/end-to-end latency, and ONNX CPU inference on a cooled Raspberry Pi 5. The central empirical claim is that, with the full 2 800-image training set, mAP50 differs by only 1.04 points across all twelve variants, so architectural generation alone does not determine final accuracy; instead data availability, target hardware and required FPS must be considered jointly. YOLOv8 is most data-efficient (≥90 % mAP50 at 400 images), while YOLO26n is fastest on the Pi 5 (7.51 FPS) and YOLOv5mu wins among medium models on CPU.","tokens_in":19443,"tokens_out":1136,"duration_ms":22471,"significance":"If the tabulated rankings hold, the work supplies a concrete, deployment-oriented decision framework for practitioners choosing lightweight detectors for on-farm aquaculture edge devices. Strengths include a fair Ultralytics-only comparison under identical hyper-parameters and fixed splits, dual-platform (A100 + Raspberry Pi 5) timing that exposes GPU-to-CPU rank reversals, systematic learning-curve analysis, and the promised public release of RAS-MortDB plus all twelve .pt/.onnx weights. These elements make the modest claim falsifiable and immediately useful beyond the single-species mortality use-case.","major_comments":[{"comment":"Table 3 and §3.1.1–3.1.2: the headline claim that architectural generation has “little influence” rests on a 1.04-point mAP50 spread obtained from single training runs (fixed seed 42, one restart per configuration). The authors themselves note that differences “fall well within normal run-to-run training variance,” yet no multi-seed standard deviations or confidence intervals are reported. Without at least three independent full-dataset restarts (or bootstrap estimates), it is impossible to judge whether the observed spread is statistically smaller than training stochasticity; the same caveat applies to the learning-curve thresholds in Figure 5.","section":"Table 3 / §3.1"},{"comment":"§2.5 and Table 5: edge rankings (and the claimed validation of YOLO26’s CPU advantage) are derived from a single actively-cooled Raspberry Pi 5 running ONNX Runtime only. Absolute FPS and the medium-tier reversal (YOLOv5mu > YOLO26m) may change under TFLite, OpenVINO, different ARM SoCs, or thermal throttling. Because the paper’s joint-selection recommendation hinges on hardware-specific behaviour, a second device or runtime would materially strengthen the deployment claim; the current single-platform result remains a useful but narrow data point.","section":"§2.5 / Table 5"}],"minor_comments":[{"comment":"Inconsistent naming: the text alternates between “YOLO11” and “YOLOv11” (and occasionally “YOLOv5” vs “YOLOv5u”). Standardise on the Ultralytics designations used in Table 1.","section":"throughout"},{"comment":"Figure 4 panels lack error bars or shaded ranges; even if only single runs were performed, a note in the caption would help readers interpret the apparent saturation after 1 000 images.","section":"Figure 4"},{"comment":"Minor typos: “sea food” (p. 2), “intensi ve”, “Fas ter R-CNN”, and a duplicated “Jocher, G.; Qiu, J.” in reference 33. Also, several “accessed on 15 April 2026” dates appear future-dated relative to typical arXiv practice.","section":"Introduction / References"},{"comment":"Table 4 residual-latency discussion (§3.2) would be clearer if the exact formula Avg − Infer − Post were stated once and the values tabulated.","section":"§3.2 / Table 4"}],"recommendation":"minor_revision","confidential_remarks":"Solid, well-scoped empirical contribution that fits aquaculture-engineering or applied-CV venues. The single-seed and single-device limitations are real but already partially acknowledged; they do not invalidate the comparative rankings. No integrity or novelty concerns."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is a clean engineering benchmark, not a methods paper. The new piece is the first peer-reviewed head-to-head of YOLO26 (NMS-free, edge-oriented) against YOLOv5u/v8/11 on a real recirculating-aquaculture mortality task, with learning curves and dual-platform timing (A100 + Raspberry Pi 5 ONNX). That is worth having.\n\nWhat they do well: twelve models, seven dataset sizes, fixed splits, and complementary metrics (mAP50/50-95, Infer/Post/Avg latency, ONNX size, FPS). The central result is clear and modest: with the full 2,800-image set, mAP50 spreads only 1.04 points, so generation barely moves final accuracy. Data efficiency and hardware rankings do move—YOLOv8 hits 90% mAP50 at 400 images while YOLO26n/s need ~1,000; YOLO26n leads Pi 5 nano FPS (7.51); YOLOv5mu wins the medium CPU tier. That joint-selection claim is directly supported by the tables. Methods are transparent, self-citations to MortCam are appropriate, and they flag the single-tank / adult-salmon / cull-augmented limits themselves.\n\nSoft spots are real but secondary. No multi-seed variance or error bars; one Pi 5 and ONNX only; high-mortality frames come from euthanized culls on the last day. Those constrain transfer claims, not the internal rankings. Training defaults and COCO pretraining are standard, not circular. Citation pattern is fine for an applied CV paper.\n\nWho it is for: people actually choosing detectors for on-farm edge boxes in aquaculture (or similar constrained vision). Not for anyone hunting architectural novelty. I would send it to peer review; it is reproducible enough and the evidence matches the claim. Worth a reading-group slot if the group cares about deployment trade-offs; I would cite the Pi 5 and data-efficiency numbers when I next write about edge selection in this domain.","headline":"Solid, useful edge-deployment benchmark of YOLO26 on real RAS mortality data; architecture alone is not the story, and the numbers back that up.","tokens_in":19998,"tokens_out":522,"would_cite":true,"duration_ms":5973,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Architectural novelty alone does not pick the best YOLO for edge aquaculture: accuracy converges, data and hardware decide.","keywords":["YOLO","fish mortality detection","edge deployment","precision aquaculture","recirculating aquaculture systems","NMS-free inference","learning curve","Raspberry Pi"],"falsifier":"Repeat the identical training-size and Raspberry Pi ONNX benchmark on a multi-tank, multi-species, multi-life-stage mortality dataset that includes natural high-mortality outbreaks; if YOLO26 nano/small then match or beat YOLOv8 at 400 images, or if medium-tier CPU rankings reverse, the joint-selection claim collapses for those conditions.","tokens_in":20180,"feed_emoji":"🐟","tokens_out":1089,"duration_ms":9259,"temperature":0.7,"pith_summary":"This paper asks whether the newest YOLO26 detector, built for NMS-free end-to-end inference on CPU edge devices, actually beats its Ultralytics predecessors for a practical aquaculture task: spotting dead versus live fish in a recirculating tank. Twelve model variants (nano, small, medium across four generations) were trained and tested on the same mortality images, then timed on both a high-end GPU and a Raspberry Pi 5. On the full dataset every model lands within about one percentage point of mAP50, so generation barely moves final accuracy once enough labeled images exist. The real differences appear elsewhere: YOLOv8 reaches 90 percent mAP50 with only 400 images while YOLO26 nano and small need roughly 1,000, yet YOLO26n is the fastest on the Pi (7.51 FPS) and YOLOv5mu wins the medium-tier CPU race. The authors therefore argue that farms should choose models by joint constraints of data volume, target hardware, and required frame rate rather than by release date.","feed_headline":"Newest YOLO is not always best for edge fish monitoring","feed_subtitle":"Accuracy converges across generations; data volume and Pi speed decide the winner","key_machinery":"A controlled twelve-model, seven-dataset-size learning-curve benchmark that measures mAP50, training time, and end-to-end latency on both an A100 GPU and an ONNX Runtime Raspberry Pi 5, isolating the effect of NMS-free one-to-one assignment versus earlier NMS-based heads under identical aquaculture data.","core_discovery":"When twelve YOLO variants spanning four generations and three size tiers are trained on the same full fish-mortality dataset, final detection accuracy differs by only 1.04 percentage points of mAP50; architectural generation therefore exerts little influence on accuracy once training data are plentiful. Meaningful trade-offs instead appear in data efficiency (YOLOv8 reaches 90 percent mAP50 with 400 images; YOLO26 nano/small need 1,000) and in CPU edge speed (YOLO26n leads the nano tier at 7.51 FPS on a Raspberry Pi 5; YOLOv5mu leads the medium tier).","pith_inferences":["The same data-efficiency gap may appear in other sparse-object aquaculture tasks (feed pellets, early-stage juveniles) that also rely on one-to-one assignment heads.","If ONNX Runtime is replaced by TensorRT or TFLite on the same Pi-class hardware, the relative CPU ranking of NMS-free versus NMS-based heads could shift and should be re-measured.","Commercial operators running many tanks may find that the storage and initialization advantages of the smallest YOLO26n ONNX export matter more than raw FPS once multi-camera fleets are considered.","The finding that GPU speed orderings reverse on CPU hardware is likely generalizable beyond aquaculture to any edge-vision domain that still ships NMS-based detectors."],"forward_implications":["Farms with limited labeled mortality images should prefer YOLOv8 over YOLO26 for faster convergence to operational accuracy.","CPU-only edge boxes monitoring tanks can adopt YOLO26n for the highest nano-tier frame rate without sacrificing final mAP50 once data are abundant.","GPU-only ranking tables cannot be trusted as proxies for Raspberry-Pi or other ARM-CPU deployment choices; target-hardware timing is required.","Annotation effort beyond roughly 1,000 images yields diminishing accuracy returns for this binary dead/live task, so resources are better spent on image quality or scenario diversity.","Model selection checklists for aquaculture edge AI should list data volume, hardware class, and required FPS as co-equal criteria with architectural generation."],"fun_headline_variants":["YOLO26 tops Pi speed but needs more data than YOLOv8 for fish detection","Accuracy nearly ties across four YOLO gens once training data is plentiful","Data volume and Raspberry Pi FPS trump architectural novelty for edge fish AI","YOLO26n hits 7.51 FPS on Pi 5; YOLOv8 reaches 90% mAP50 with just 400 images","Newer YOLO not always best: hardware and data set the real edge winners"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The accuracy and speed rankings measured on one Atlantic-salmon grow-out tank, under two lighting regimes, with high-mortality scenes created by adding euthanized culls on the final day, are assumed representative enough of real commercial mortality and other species or life stages for the selection guidance to transfer.","fun_headline_variants_meta":{"raw":{"variants":["YOLO26 tops Pi speed but needs more data than YOLOv8 for fish detection","Accuracy nearly ties across four YOLO gens once training data is plentiful","Data volume and Raspberry Pi FPS trump architectural novelty for edge fish AI","YOLO26n hits 7.51 FPS on Pi 5; YOLOv8 reaches 90% mAP50 with just 400 images","Newer YOLO not always best: hardware and data set the real edge winners"]},"model":"grok-4.5","effort":"low","cost_usd":0.004192,"raw_usage":{"total_tokens":1370,"prompt_tokens":911,"num_sources_used":0,"completion_tokens":97,"cost_in_usd_ticks":41920000,"prompt_tokens_details":{"text_tokens":911,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":362,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":911,"tokens_out":97,"duration_ms":3872,"temperature":1.0,"reasoning_tokens":362,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T15:11:06.993263+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Repeat the identical training-size and Raspberry Pi ONNX benchmark on a multi-tank, multi-species, multi-life-stage mortality dataset that includes natural high-mortality outbreaks; if YOLO26 nano/small then match or beat YOLOv8 at 400 images, or if medium-tier CPU rankings reverse, the joint-selection claim collapses for those conditions.","supporting_citations":[],"review_version":1}