{"id":"919f6a70-98fa-4833-b74d-1cbd13f73928","arxiv_id":"2504.13099","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"On an 857-image orchard dataset, RF-DETR achieved the best mAP@50 for both single-class (0.9464) and multi-class (0.8298) greenfruit detection, while YOLOv12N and YOLOv12L led mAP@50:95 in the two settings.","lead":"This paper compares a transformer-based object detector (RF-DETR) with a CNN-based detector (YOLOv12) on a custom dataset of green apples in a commercial orchard. It reports that RF-DETR wins at the common mAP@50 metric while some YOLOv12 variants win at stricter overlap, information that could guide model choice for agricultural robots.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Training protocols are internally contradictory: §2.3.1 gives RF-DETR batch 8/300 epochs; §2.4 gives 50–100 epochs at batch ~16, vs 100 for YOLOv12. The reported mAP gap may reflect training budget, not architecture.","rationale":"I agree with the reader's weakest_assumption: the paper's own methods sections undermine the claim of identical experimental settings. §2.3.1 and §2.4 describe different batch sizes and epoch counts for RF-DETR, while YOLOv12 variants are consistently described as trained for 100 epochs; §2.5 then asserts equal training protocols. This is an internal inconsistency, not a dispute with external consensus, and it strikes directly at the causal interpretation of the mAP table. If the training budgets differ, the reported mAP advantage could simply reflect different amounts of optimization rather than architectural superiority. The convergence analysis in §3.3 inherits the same problem, since the models' learning curves are compared over different training lengths. The absence of released code, data, and error bars compounds the issue, but the central flaw is the unsubstantiated fairness premise. I would keep the reader's CONDITIONAL verdict: the claim is plausible and the dataset is real, but the comparison as reported is not verifiable until the authors align training protocols, provide repeated runs, and release the data/code needed to reproduce the table. No adjustment to the reader's verdict is needed.","tokens_in":17624,"tokens_out":6539,"duration_ms":57781,"concrete_test":"Run a controlled re-benchmark on the same fixed train/validation split: train RF-DETR-Base and YOLOv12X/L/N for exactly 100 epochs with batch size 16 and each framework's stated optimizer/LR schedule, and additionally train RF-DETR for 300 epochs at batch 8 as §2.3.1 describes. Repeat with three seeds and report mAP@50 with 95% CIs. If RF-DETR still exceeds YOLOv12 by more than the CI, the architecture claim survives; if the gap shrinks or reverses under equal budgets, the headline mAP claim is unsubstantiated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that RF-DETR achieves higher mAP@50 (0.9464 single-class; 0.8298 multi-class) than YOLOv12 under 'identical experimental settings' (§2.4, reiterated in §2.5). That premise is contradicted by the paper's own methods: §2.3.1 states RF-DETR was trained with batch size 8 for 300 epochs, while §2.4 states RF-DETR was trained for 50 epochs (single-class) and 100 epochs (multi-class) with a batch size of approximately 16, and all YOLOv12 variants were trained for 100 epochs. If these descriptions reflect the actual runs, the models were not given equal training budgets, optimizer steps, or batch sizes, so the observed mAP differences cannot be attributed to architecture. The convergence conclusion in §3.3 is similarly affected: showing RF-DETR 'plateauing within 10 epochs' while YOLOv12 trains for 100 epochs is not a controlled comparison. The paper also provides no error bars or repeated-seed variance, so the point estimates are not shown to be stable. The most load-bearing weakness is therefore not disagreement with any external benchmark but an internal inconsistency that invalidates the fairness premise on which the headline comparison depends.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports an empirical comparison of RF-DETR-Base (a transformer-based detector) and three YOLOv12 variants (X, L, N; CNN-based) for detecting immature green apples in a commercial orchard. The authors constructed a custom dataset of 857 RGB images with two labeling schemes: a single-class scheme (greenfruit) and a multi-class scheme (occluded vs. non-occluded greenfruit). They report precision, recall, F1-score, mAP@50, and mAP@50:95 for each model and task, and they analyze training convergence. The headline results are that RF-DETR achieves the highest mAP@50 in both single-class (0.9464) and multi-class (0.8298) detection, while YOLOv12N achieves the highest single-class mAP@50:95 (0.7620) and YOLOv12L the highest multi-class mAP@50:95 (0.6622). The authors conclude that RF-DETR excels at localizing occluded and camouflaged fruit, while YOLOv12 offers advantages in speed-oriented deployments. The paper also includes qualitative examples and a discussion of label ambiguity.","tokens_in":17888,"tokens_out":2472,"duration_ms":21985,"significance":"If the reported results were obtained under truly identical training protocols and were statistically stable, the paper would provide a useful apples-to-apples benchmark of two popular detection paradigms in a challenging precision-agriculture scenario, with a new field dataset and a practical occlusion-labeling scheme. The qualitative observations about RF-DETR handling occlusion and camouflage are plausible and align with the architectural differences between DETR-style and YOLO-style detectors. However, the paper currently ships no code, no data, no dataset splits, and no uncertainty estimates, and the methods sections contradict each other on the training protocols. As a result, the central comparative claim is not yet substantiated. The work has potential value as an application study, but its current form is not a reliable basis for architecture-level conclusions.","major_comments":[{"comment":"The training protocols are internally inconsistent. Section 2.3.1 states that RF-DETR was trained with a batch size of 8 for 300 epochs, while Section 2.4 states that RF-DETR was trained for 50 epochs on the single-class dataset and 100 epochs on the multi-class dataset with a batch size of approximately 16, and that all YOLOv12 models were trained for 100 epochs. These descriptions cannot both be true. Since Section 2.4 and Section 2.5 claim that both models were trained under 'identical experimental settings,' the contradiction directly undermines the fairness premise of the entire comparison. The authors must clarify the exact training budget (epochs, batch size, optimizer steps, learning rate schedule) actually used for each model and dataset, or the headline mAP differences cannot be attributed to architecture.","section":"Section 2.3.1 vs. Section 2.4"},{"comment":"All performance metrics are reported as single-run point estimates with no error bars, confidence intervals, or repeated-seed variance. The paper treats differences as small as 0.01 (e.g., multi-class mAP@50:95 of 0.6609 for YOLOv12X vs. 0.6530 for RF-DETR, Section 3.2) as meaningful. Without variance estimates or a significance test, these differences are not distinguishable from noise, especially given the small dataset of 857 images and the label-ambiguity issues the authors themselves describe. The authors should provide at least three repeated runs per model and report mean and standard deviation, or justify why a single run is sufficient.","section":"Section 2.5 and Table 1"},{"comment":"The dataset construction section gives no train/validation/test split information. The authors report 857 images and 4,125 object labels for the single-class scheme but do not state how many images were used for training, validation, and testing, whether split was performed at the image level, or whether any images from the same orchard row or session could leak between splits in a way that inflates metrics. Without this information, the reported mAP numbers cannot be interpreted, and the comparison between models may be affected by split-specific variance. This is a load-bearing omission for a benchmark paper.","section":"Section 2.2"},{"comment":"The claim that RF-DETR 'is the only model to surpass 60% mAP on the COCO dataset to date' (Section 1.2, restated in Section 2.3.1) is factually incorrect and is even contradicted by the same paper, which states that YOLOv12X achieves 68.9 mAP on COCO (Section 2.3.2). This overstatement should be removed or replaced with an accurate citation-specific claim about transformer-based real-time detectors, if that is what is intended.","section":"Section 1.2 and Section 2.3.1"}],"minor_comments":[{"comment":"The convergence analysis compares RF-DETR (trained for 50 epochs in single-class) with YOLOv12X (trained for 100 epochs). Presenting RF-DETR as 'plateauing within 10 epochs' while YOLOv12 continues to train is misleading when the training budgets differ; the section should at least acknowledge this asymmetry or use an equal-iteration comparison.","section":"Section 3.3"},{"comment":"The index terms and abstract mention YOLOv13, YOLOv14, YOLOv15, YOLOE, and YOLO World, none of which are evaluated or discussed in the paper. These terms should be removed or properly cited if they are meant to be related work.","section":"Section 1.2 and Index Terms"},{"comment":"There are numerous typographical and formatting errors, including 'comparision' (Section 3.2 caption), 'e ffective' in several places, and inconsistent spacing in equations and references. A thorough copyedit is needed.","section":"Throughout"},{"comment":"The label ambiguity discussion is qualitative and does not quantify inter-annotator agreement. Since the paper's multi-class task hinges on the distinction between occluded and non-occluded fruit, reporting a metric such as Cohen's kappa or a per-image ambiguity rate would strengthen the reliability of the ground truth.","section":"Section 2.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is borderline between major revision and reject. The central claim—that RF-DETR outperforms YOLOv12 under identical settings—is currently unsupported by the internal contradictions in the methods and the lack of statistical grounding. However, these issues are fixable: the authors can clarify the actual training protocols, provide dataset splits, and add variance estimates. If the authors are unwilling or unable to provide those details, the manuscript should not be accepted. I would also note that the paper has a very heavy self-citation pattern (refs [17], [31], [34], [49], [50], [51], and [53] are all from the same group), which may warrant editorial attention."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: this is a benchmark paper with a plausible result—RF-DETR beats YOLOv12 on mAP@50 for greenfruit detection in a new orchard dataset—but the methods sections contradict each other on the training protocol, so the headline fairness claim doesn't hold as written.\n\nWhat's new: it's the first direct RF-DETR vs YOLOv12 comparison I've seen on agricultural data. The custom dataset of 857 images with single-class and occluded/non-occluded multi-class labels is a useful resource, and the qualitative examples in Figures 5 and 6 show RF-DETR handling heavy occlusion and camouflage where YOLOv12 misses. The convergence observation (Figure 8) is interesting, though it's entangled with the protocol mess.\n\nThe soft spots are real. Section 2.3.1 says RF-DETR was trained with batch size 8 for 300 epochs. Section 2.4 says RF-DETR got 50 epochs (single-class) or 100 (multi-class) at batch size ~16, while all YOLOv12 models got 100 epochs. Section 2.5 then claims 'identical experimental settings.' These three statements cannot all be true. Either the authors ran different budgets and didn't report them correctly, or they sloppily copied defaults into the paper. Either way, the mAP differences can't be attributed to architecture without knowing the actual protocol. The convergence claim—RF-DETR 'plateaus in under 10 epochs' while YOLOv12 trains for 100—is not a controlled comparison.\n\nAlso missing: error bars, repeated runs, dataset split details, and code/data release. The mAP@50:95 margins are narrow (0.7433 vs 0.7620 for single-class), so single-run point estimates don't support strong 'outperforms' language. The abstract lists YOLOv13/14/15 index terms that don't appear in the paper, which suggests the metadata was copy-pasted.\n\nWho this is for: precision-ag practitioners looking for a model-selection data point, not methods readers. The dataset and the qualitative findings could be valuable if the authors clean up the protocol description, release the data/code, and add variance information.\n\nMy recommendation: send to peer review with a clear expectation of major revision. The comparison is relevant and the data are real, but the current manuscript is not independently verifiable because of the internal contradictions and missing artifacts. A referee could help the authors turn this into a usable benchmark paper.","headline":"A useful dataset and an interesting RF-DETR vs YOLOv12 comparison, but the paper's own methods sections contradict each other on training protocols, so the headline fairness claim doesn't hold as written.","tokens_in":18404,"tokens_out":4084,"would_cite":false,"duration_ms":32763,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RF-DETR achieves the highest mAP@50 for greenfruit detection in single-class and multi-class orchard settings, while YOLOv12N leads the stricter mAP@50:95 in single-class.","keywords":["greenfruit detection","RF-DETR","YOLOv12","transformer object detection","CNN object detection","occlusion handling","label ambiguity","orchard environment"],"falsifier":"Retrain both model families on the same split with identical batch size, optimizer, and epoch count, or a fixed compute budget; if YOLOv12X closes or reverses the mAP@50 gap, the claimed architectural advantage is unsupported. A second check would measure inter-annotator agreement on the occluded/non-occluded labels, since low agreement would weaken the multi-class ranking.","tokens_in":17419,"feed_emoji":"🍏","tokens_out":9029,"duration_ms":72377,"temperature":0.7,"pith_summary":"This paper compares RF-DETR, a transformer-based detector that combines a DINOv2 backbone with deformable attention, against three YOLOv12 CNN variants for detecting immature green apples in a commercial orchard where fruits blend into foliage and are often partially hidden. On a custom set of 857 images with single-class and occluded/non-occluded multi-class labels, RF-DETR attains the highest mAP@50 in both tasks (0.9464 and 0.8298), while YOLOv12N attains the highest single-class mAP@50:95 (0.7620) and YOLOv12L the highest multi-class mAP@50:95 (0.6622). The paper also reports that RF-DETR converges within a small number of epochs, far sooner than YOLOv12X. The aim is to show that the transformer's global context modeling is better suited to occlusion and camouflage in precision agriculture, with YOLOv12 remaining competitive under stricter overlap thresholds and for speed-sensitive deployment.","feed_headline":"Transformer beats YOLOv12 on greenfruit mAP@50","feed_subtitle":"RF-DETR hits 0.9464 mAP@50 in cluttered orchards; YOLOv12N still leads at stricter overlap thresholds.","key_machinery":"The object that carries the comparison is the architectural pair: RF-DETR-Base, a real-time detection transformer combining a DINOv2 backbone, deformable cross-attention, single-scale features, and NMS-free set prediction, versus YOLOv12, a CNN detector built on R-ELAN blocks, 7x7 separable convolutions, and area attention. The paper uses this pair to attribute detection differences to global context in the transformer versus local feature extraction in the CNN. The measured instruments are the standard detection metrics precision, recall, F1, mAP@50, and mAP@50:95, with mAP@50 serving as the headline comparison and mAP@50:95 as the stricter-overlap check.","core_discovery":"On its own terms, the paper's central discovery is that the transformer-based RF-DETR-Base model outperforms all tested YOLOv12 configurations at the 50% IoU threshold for greenfruit detection in a cluttered orchard, with single-class mAP@50 of 0.9464 and multi-class mAP@50 of 0.8298. The best YOLOv12 variant, YOLOv12N, surpasses RF-DETR at the stricter mAP@50:95 metric in single-class detection (0.7620 versus 0.7433), and YOLOv12L does the same in multi-class detection (0.6622 versus 0.6530). Qualitatively, RF-DETR detects heavily occluded and camouflaged fruitlets that YOLOv12 misses, and it classifies occlusion status more accurately in the multi-class case. The paper interprets these results as evidence that global context modeling from transformer attention is the decisive factor in label-ambiguous orchard scenes.","pith_inferences":["The reported mAP gap is not necessarily architectural, because the methods give the two model families different training budgets; a matched-compute rerun would test whether the gap persists.","The occlusion labels are inherently subjective, so the multi-class comparison would be stronger with inter-annotator agreement measured; low agreement would mean part of the occlusion-classification gap is label noise.","RF-DETR's fast convergence hints that transformer-based detectors could be trained effectively with far fewer annotated orchard images, a testable extension for other fruit crops."],"forward_implications":["If RF-DETR's margin at mAP@50 holds, accuracy-critical tasks such as robotic thinning and early yield estimation should prefer the transformer model over YOLOv12 on similar orchard data.","The fast convergence of RF-DETR means domain-specific agricultural detectors can reach stable performance in tens of epochs, cutting compute costs in repeated orchard surveys.","YOLOv12N's lead at mAP@50:95 in single-class detection keeps CNN detectors in play for applications that demand tight box alignment or edge deployment.","The multi-class occlusion result implies that transformer detectors may reduce false negatives on partially hidden fruit, which is the main counting error in dense canopies."],"supporting_citations":[{"why":"Defines YOLOv12's architecture, the CNN baseline the paper compares against.","marker":"[32]"},{"why":"Supplies the deformable attention mechanism that RF-DETR inherits for selectively attending to relevant spatial features.","marker":"[20]"},{"why":"Provides the real-time hybrid encoder idea that RF-DETR's design borrows for speed and accuracy.","marker":"[43]"},{"why":"Collaborative hybrid assignments training that RF-DETR adopts for stability under label ambiguity.","marker":"[40]"},{"why":"Establishes the end-to-end, NMS-free set prediction framework that RF-DETR builds on.","marker":"[22]"},{"why":"Reports the benchmark results used to claim RF-DETR surpasses earlier YOLO models and motivates the comparison.","marker":"[48]"}],"fun_headline_variants":["RF-DETR beats YOLOv12 on greenfruit mAP@50","Transformers win at mAP@50 for cluttered orchard greenfruit","RF-DETR leads mAP@50, YOLOv12N leads mAP@50:95","RF-DETR converges fast, tops YOLOv12 on mAP@50","Global context beats CNN on greenfruit mAP@50"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the models were compared under identical experimental settings; the paper itself reports different training epochs and batch sizes for RF-DETR and YOLOv12, so if those differences matter, the architecture comparison is not clean.","fun_headline_variants_meta":{"raw":{"variants":["RF-DETR beats YOLOv12 on greenfruit mAP@50","Transformers win at mAP@50 for cluttered orchard greenfruit","RF-DETR leads mAP@50, YOLOv12N leads mAP@50:95","RF-DETR converges fast, tops YOLOv12 on mAP@50","Global context beats CNN on greenfruit mAP@50"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000793,"raw_usage":{"total_tokens":3611,"prompt_tokens":1178,"completion_tokens":2433,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":794,"completion_tokens_details":{"reasoning_tokens":2327}},"tokens_in":794,"tokens_out":2433,"duration_ms":14755,"temperature":1.0,"reasoning_tokens":2327,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:14:05.474443+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain both model families on the same split with identical batch size, optimizer, and epoch count, or a fixed compute budget; if YOLOv12X closes or reverses the mAP@50 gap, the claimed architectural advantage is unsupported. A second check would measure inter-annotator agreement on the occluded/non-occluded labels, since low agreement would weaken the multi-class ranking.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the real-time hybrid encoder idea that RF-DETR's design borrows for speed and accuracy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Collaborative hybrid assignments training that RF-DETR adopts for stability under label ambiguity."}],"review_version":1}