{"id":"396a2ed0-21ea-499f-99bb-58c06d064cd9","arxiv_id":"2412.02642","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A large-scale field study of robot-video soybean seed counting shows good seed-count accuracy but essentially no correlation between predicted and harvested plot yield on the full test set.","lead":"This paper tests a ground-robot video pipeline that counts soybean seeds and predicts plot yield for breeding trials. The seed counting works, but the yield predictions are nearly uncorrelated with harvested yield on the full test set and the headline ranking accuracy is below a trivial baseline.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim fails on the paper's own data: P2PNet-Yield shows R²=0.0062 on the full 650-plot field (Fig. 11a), and the 83% ranking accuracy is below the always-select-none baseline at the 10% threshold.","rationale":"In good faith, the seed-counting component has some independent support: the MIX AUG model's MAE=20.54 and R²=0.87 on the image test set are internally consistent, and the augmentation ablations move in the expected direction. The problem is the yield-estimation step, which is the paper's central claim. For a scalable breeding tool, the yield regressor must work on the population of plots a breeder actually has; Section 3.3 shows it only achieves R²=0.38 after manually removing lodging, disease, and imaging anomalies, while Figure 11a shows R²=0.0062 on the full 650-plot dataset. The 83% ranking accuracy in the abstract is computed on an imbalanced 10% selection problem and is actually below the always-select-none baseline of 90%, so it cannot rescue the claim. The 32% time/cost reduction has no supporting cost model, and the undisclosed training protocol plus lack of code/data prevents checking for leakage. These are internal failures, not disagreements with external consensus; the paper's own reported numbers undermine its core contribution. I therefore keep the reader's REJECT verdict and see no need to adjust it.","tokens_in":16575,"tokens_out":8470,"duration_ms":86353,"concrete_test":"Obtain the trained P2PNet-Yield model (or retrain it per §2.3), then run it on all 650 plots of the 2023 F7 data without excluding any anomalous plots, using the same 20-image sampling and spatial adjustment described in §2.1.4–§2.1.2. Report R², MAE, and the 10/20/30% selection metrics against spatially adjusted ground truth, and compare the 10% accuracy to the always-select-none baseline of 585/650 = 90%. If full-dataset R² is ≈0.0062 or the 10% accuracy does not exceed 90%, the yield-estimation and genotype-ranking claims are unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that P2PNet-Yield estimates plot seed yield well enough to rank soybean genotypes in a breeding program. For that claim to hold, the Yield Regression Module trained on a manually curated, anomaly-free 100-plot subset must generalize to the other plots in the 2023 field. Section 3.3 and Figure 11a show it does not: on the full 650-plot dataset, R² between estimated and harvested yield is 0.0062, i.e. essentially no predictive relationship. The only positive result (R²=0.38, MSE=6.53) comes from 100 test plots selected by removing lodging, disease, large-gap, and bad-imaging anomalies. A method that works only after manual curation is not a scalable solution for routine breeding trials, where such anomalies are common. In addition, the headline 'genotype ranking accuracy up to 83%' is not evidence of ranking ability: at the 10% threshold, always selecting no plot gives 585/650 = 90.0% accuracy, above the reported 542/650 = 83.4%. The same is true at 20% (always-negative baseline 80% vs 70% reported) and 30% (70% vs 60% reported). The claimed 32% time/cost reduction has no supporting cost analysis, and neither code nor data was released, so the undisclosed yield-regression training protocol cannot be checked for leakage or in-sample evaluation. The load-bearing assumption—generalization from a curated subset to an actual breeding field—fails on the paper's own reported numbers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes a pipeline for soybean yield estimation from ground-robot fisheye videos. Images are corrected, cropped, and processed by P2PNet-Yield, which combines the P2PNet-Soy feature-extraction backbone with a small regression head to predict plot yield (t/ha). The seed-counting component is trained and evaluated on 2021 and University of Tokyo data with camera-sensor augmentation, achieving its best result on the MIX AUG combination (MAE 20.54, R²=0.87). The yield regression module is trained and tested only on a manually curated 100+100 subset of 2023 plots, and the paper reports a genotype ranking accuracy of up to 83% and a 32% reduction in time and cost for yield data collection.","tokens_in":16887,"tokens_out":5496,"duration_ms":53074,"significance":"If the yield-estimation claim held, the work would be a valuable step toward non-destructive, high-throughput yield phenotyping for soybean breeding. The seed-counting contribution is a genuine engineering achievement: the MIX AUG model shows a substantial improvement in MAE over the non-augmented baselines, and the creation of a large field-image dataset is a useful resource for the community. However, the central yield-estimation and ranking claims are not supported by the paper's own results. The full-dataset correlation between estimated and harvested yield is effectively zero, and the reported ranking accuracy is below a trivial baseline. The positive yield result is confined to a manually curated, anomaly-free subset and does not generalize to the uncurated field data, which is the actual target of the claimed scalable solution.","major_comments":[{"comment":"The central claim of the paper—that P2PNet-Yield estimates plot seed yield well enough to support breeding decisions—fails on the full 650-plot 2023 dataset. Figure 11a reports R²=0.0062 between estimated and harvested yield, i.e., essentially no predictive relationship. The manuscript itself states in Section 3.3 that 'no correlation was noted in the uncurated dataset.' The only positive result (R²=0.38, MSE=6.53) comes from 100 test plots obtained by manually removing plots with lodging, disease, large gaps, and bad imaging. A method that requires such manual curation is not a 'robust' or 'scalable solution' as claimed in the abstract and conclusion.","section":"Section 3.3, Figure 11a"},{"comment":"The headline 'genotype ranking accuracy up to 83%' is not evidence of ranking ability. At the 10% selection threshold, the reported accuracy is 542/650 = 83.4%, but the always-select-none baseline gives 585/650 = 90.0%. The same pattern holds at 20% (reported 70% vs. 80% baseline) and 30% (reported 60% vs. 70% baseline). The low sensitivity values (0.17, 0.25, 0.33) further confirm that the model is not identifying top-performing lines. Thus the accuracy metric, as presented, is misleading and does not support the abstract's claim of effective genotype ranking.","section":"Section 3.2.2, Table 2"},{"comment":"The near-zero correlation (R²=0.06) between estimated total seed count (TSC) and estimated yield is a red flag for the yield regression module. Since both outputs derive from the same feature-extraction backbone applied to the same 20 sample images per plot, one would expect them to be strongly related if the regression were tracking seed information. The near-zero correlation suggests the yield regression is not consistently using the seed-count signal, further undermining the validity of the estimated yields used in the ranking analysis.","section":"Figure 8"},{"comment":"The claim of 'up to a 32% reduction in time to collect yield data as well as costs associated with traditional yield estimation' is unsupported. No time-motion study, cost model, or any quantitative comparison of the proposed pipeline with combine harvesting appears anywhere in the methods or results. This is a quantitative claim in the abstract and conclusion that cannot be verified from the manuscript.","section":"Abstract and Section 5"},{"comment":"The yield regression module is trained and evaluated on a single manually curated 100-plot subset from one 2023 field, with no independent validation set from a different year, location, or even from the uncurated portion of the same field. The failure on the full 2023 dataset (Figure 11a) shows that the module does not generalize from the curated subset to the actual breeding-trial population. The manuscript's own limitation statement in Section 4, acknowledging the model's dependence on image quality and the experience-based sampling/fusion choices, does not address this more fundamental generalization failure.","section":"Sections 2.3, 3.3"}],"minor_comments":[{"comment":"The text says the ISU AUG model has R²=0.87 in Figure 6c, but the Figure 6c caption states R²=0.78; the R²=0.87 value belongs to Figure 6d (MIX AUG). This inconsistency should be corrected.","section":"Section 3.1, Figure 6"},{"comment":"The sentence 'MIX NO AUG and ISU AUG had a similar performance with slightly better MSE and MAE for ISU NO AUG (Table 1)' appears to be a typo: Table 1 shows MIX NO AUG, not ISU NO AUG, has slightly better MSE and MAE than ISU AUG.","section":"Section 3.1, Table 1"},{"comment":"The abstract lists three years of plot data (8500 in 2021, 2275 in 2022, 650 in 2023), but Section 2.1.1 describes robot video data collected only in 2021 and 2023, and no 2022 data appear in the methods or results. This discrepancy should be resolved.","section":"Abstract, Section 2.1.1"},{"comment":"The axes of Figure 11 are labeled with yield values in the range 50–100, whereas the text reports yields in t/ha (with typical values around 5–8 t/ha in Figure 8). The units in Figure 11 are undefined and likely inconsistent with the rest of the paper; please clarify whether these are bushels/acre or some other unit.","section":"Figure 11"},{"comment":"The description of image sampling is confusing: the text first says each row is divided into eight sections with seven splitters and the middle five are chosen, then says 'The two rows of the same plot were treated as a single row, resulting in ten images per side and twenty images per plot.' The arithmetic behind these numbers should be explained more clearly.","section":"Section 2.1.4"},{"comment":"The data availability statement provides only an email address for requesting data. Since the paper does not release code or model weights, the yield-regression training protocol cannot be independently reproduced; please consider providing at least the trained model weights and the exact data split used in Section 3.3.","section":"Data Availability Statement"}],"recommendation":"reject","confidential_remarks":"The seed-counting portion of the paper (Section 3.1, Table 1) is a useful empirical contribution, and the dataset collection effort is substantial. However, the yield-estimation and ranking claims, which are the paper's stated central contributions, are contradicted by the full-dataset results in Figure 11a and by the baseline comparison in Table 2. These are load-bearing issues rather than presentation problems, and fixing them would require new experiments and a fundamentally different framing of the paper's contribution. I would also note that the abstract's three-year claim appears inconsistent with the methods, which describe only 2021 and 2023 data; this should be corrected in any future submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper has a genuinely useful seed-counting subresult and a yield-estimation headline that fails on the paper's own data. The seed-counting work deserves a look; the yield-ranking claims do not.\n\nWhat's new and good: a large field dataset (338,793 images from 2023, 1,200 annotated images from 2021), fisheye distortion correction, and a clean comparison of training-data combinations, including augmentation with simulated camera sensor effects. The best seed-counting model (MIX AUG) reaches MAE 20.54 and R²=0.87 on the image test set, and the augmentation ablation is internally consistent: more data plus augmentation beats less data without it. That is a solid empirical contribution for in-field seed counting.\n\nThe soft spot is the central yield-estimation claim. On the full 650-plot 2023 dataset, predicted versus harvested yield is R²=0.0062 (Figure 11a). The only positive yield result (R²=0.38) comes from a manually curated 100-plot test subset with lodging, disease, gaps, and bad imaging removed. A method that works only after that curation is not a scalable breeding tool, and the paper itself notes the uncurated correlation is zero. The abstract's 'genotype ranking accuracy up to 83%' is misleading: at the 10% threshold, always selecting no plot gives 90% accuracy, above the reported 83.4%; the same pattern holds at 20% and 30%. So the ranking metric is not evidence of ranking ability. The 32% time/cost savings has no supporting analysis, the yield-regression training protocol is undisclosed, and no code or data is released, so you can't check for leakage or in-sample evaluation. Minor inconsistency: the abstract cites 2,275 plots from 2022, but the methods only describe 2021 and 2023 fields, and the 2022 data never appears again.\n\nThe paper is not dishonest or incoherent—it shows Figure 11a and discusses limitations—but the abstract overclaims relative to what the data show.\n\nA serious referee should see this: the seed-counting component and dataset are real, and the failure of the yield model is an instructive lesson. But the yield-estimation section as written cannot be published as a scalable solution. I'd want a major revision that either drops the yield-estimation framing and presents the seed-counting contribution honestly, or demonstrates yield prediction on uncurated data. I wouldn't cite the yield claims as they stand.","headline":"The seed-counting half is a solid empirical contribution; the yield-estimation half fails on the full dataset (R²=0.0062) and the headline ranking accuracy is below the always-select-none baseline.","tokens_in":17544,"tokens_out":5346,"would_cite":false,"duration_ms":48659,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that soybean seed yield rankings can be read from ground-robot video using a deep learning pipeline, with up to 83% genotype ranking accuracy and a claimed 32% saving in time and cost over traditional combine-based data…","keywords":["soybean yield estimation","seed counting","crowd counting","deep learning","ground robot phenotyping","fisheye image correction","genotype ranking","data augmentation"],"falsifier":"Run P2PNet-Yield on all plots of a new field season with no manual curation and compare the estimated-yield rankings to combine-harvested rankings; if the rank correlation is close to the paper's full-set R2 of 0.0062 rather than the curated R2 of 0.38, the claim that the method ranks genotypes in real breeding fields is falsified.","tokens_in":16311,"feed_emoji":"🌱","tokens_out":10480,"duration_ms":94845,"temperature":0.7,"pith_summary":"This paper tries to establish that soybean seed yield can be estimated and ranked from ground-robot video rather than from combine harvesting, by counting seeds directly on mature standing plants. The authors build P2PNet-Yield, a deep learning pipeline that pairs a crowd-counting feature extractor trained to locate individual seeds with a regression head that turns the resulting feature maps into plot yield in tons per hectare. On three field seasons with 11,425 yield plots, they report genotype ranking accuracy up to 83% at a 10% selection threshold, high specificity for discarding low-yielding lines, and up to 32% reduction in time and cost for yield data collection. If true, this would let breeders screen large populations of experimental lines earlier and more cheaply, using rankings rather than absolute yield to make selection decisions.","feed_headline":"Robot video ranks soybean yields with 83% accuracy","feed_subtitle":"Deep-learning pipeline could let breeders drop low-yield soybean lines without combine-harvesting every plot.","key_machinery":"The load-bearing object is P2PNet-Yield, which combines a Feature Extraction Module with a Yield Regression Module. The Feature Extraction Module is the backbone of P2PNet-Soy, a crowd-counting model that outputs point-based seed locations; its weights are frozen during yield training. The Yield Regression Module is a small head made of one convolution layer, a max-pooling layer, and three fully connected layers; it maps the summed and concatenated feature maps from both sides of a plot to a single yield value in tons per hectare. Supporting the pipeline are fisheye-lens distortion correction followed by a 1000 by 1000 pixel center crop, random camera-sensor-effect augmentation (noise, blur, chromatic aberration, exposure), a 20-image-per-plot sampling scheme using seven splitters per row, and moving-grid spatial adjustment of both estimated and ground-truth values before ranking.","core_discovery":"On the paper's own terms, the core discovery is that seed counting on mature soybean canopies, treated as a crowd-counting problem and carried out on fisheye ground-robot video, can serve as a non-destructive proxy for yield ranking in breeding trials. The P2PNet-Yield model takes 20 sample frames per plot, passes them through the frozen P2PNet-Soy backbone, sums the per-side feature maps, concatenates them, and regresses the result to plot yield. The authors report that the MIX AUG training set, which combines their 1,200 annotated field images with an external seed-image benchmark and random camera-sensor-effect augmentation, gives the lowest counting errors and an R2 of 0.87 between predicted and annotated seed counts. For breeding decisions, ranking by estimated total seed count reaches 86% accuracy and ranking by estimated yield 83% accuracy at the 10% selection threshold, with specificity values around 0.91 to 0.92. On a manually curated subset of 100 anomaly-free 2023 plots the yield regression reaches R2 of 0.38, while on the full 650-plot set the correlation is essentially zero (R2 of 0.0062), a result the authors attribute to the model's dependence on high-quality imaging and field conditions.","pith_inferences":["Beyond the paper, the low-sensitivity/high-specificity pattern suggests practical deployment as a two-stage screen: use robot video to remove the bottom-ranked lines, then combine-harvest only the survivors.","An extension implied by the full-set R2 of 0.0062 is that the Yield Regression Module needs to be made robust to lodging, disease, and overexposure before the method can be treated as a full replacement for combine harvest; the paper itself restricts its viability claim to curated high-quality plots.","The same fisheye-correction-plus-sensor-augmentation recipe could transfer to other organ-counting tasks in the field, such as pod, fruit, or panicle counting, wherever camera variation rather than biology is the main source of error.","Since accuracy rises as the selection threshold tightens, the method fits early-generation yield trials that discard most lines; adopting it for final near-release comparisons would require sensitivity to improve."],"forward_implications":["If the ranking results hold, breeders can discard the lowest-performing experimental lines using robot video and save up to 32% of the time and cost of collecting yield data.","The P2PNet-Yield pipeline can be applied to full-sized breeding plots under field conditions: 650 plots in 2023 were imaged with side cameras and sampled as 20 representative frames per plot.","At a 10% selection threshold, ranking by estimated yield reaches 83% accuracy, and ranking by estimated total seed count reaches 86%, with specificity near 0.91 to 0.92; both methods are better at eliminating poor lines than at identifying top lines.","When trained and tested on the 100 manually curated high-quality plots, estimated yield correlates with combine yield at R2 of 0.38, indicating the architecture can track yield when imaging and field conditions are clean.","Data augmentation with random camera sensor effects reduces seed-counting error from a mean absolute error of 54.93 to 20.54 on the test set, so the counting model generalizes across cameras and lighting conditions."],"supporting_citations":[{"why":"Provides P2PNet-Soy, the seed-counting model whose backbone is reused as the Feature Extraction Module, plus the external seed-image benchmark used in training.","marker":"Zhao et al., 2023"},{"why":"Supplies the two-stage multiview fusion architecture for soybean yield estimation from ground phenotyping images that P2PNet-Yield extends.","marker":"Riera et al., 2021"},{"why":"Demonstrates high-throughput soybean pod counting with in-field robotic data collection, establishing the robot-based phenotyping approach.","marker":"McGuire et al., 2021"},{"why":"Reports the strong seed-count-to-yield correlation (r=0.92) that justifies using seed counts as a yield proxy.","marker":"Wei and Molin, 2020"},{"why":"Supplies the random camera-sensor-effect augmentation method used to improve seed-counting generalization across cameras.","marker":"Carlson et al., 2018"},{"why":"Defines the point-based crowd-counting and localization framework from which P2PNet-Soy and hence P2PNet-Yield descend.","marker":"Song et al., 2021"},{"why":"Provides the mvngGrAd moving-grid spatial adjustment, which is applied to ground-truth and estimated values before genotype ranking.","marker":"Technow, 2015"}],"fun_headline_variants":["Robot video scores soybean yield ranking at 83%","Fisheye robot video yields 83% soybean ranking accuracy","Soybean yield rank from robot videos: 83% correct","Ground robot videos cut soybean yield testing time 32%","AI counts soybean seeds from robot footage for yield"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The yield regression head is trained on 100 manually chosen, anomaly-free plots, and the whole reported ranking benefit rests on that trained head working when the same model is applied to the other 550 ordinary plots in the same field.","fun_headline_variants_meta":{"raw":{"variants":["Robot video scores soybean yield ranking at 83%","Fisheye robot video yields 83% soybean ranking accuracy","Soybean yield rank from robot videos: 83% correct","Ground robot videos cut soybean yield testing time 32%","AI counts soybean seeds from robot footage for yield"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000635,"raw_usage":{"total_tokens":3005,"prompt_tokens":1098,"completion_tokens":1907,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":714,"completion_tokens_details":{"reasoning_tokens":1825}},"tokens_in":714,"tokens_out":1907,"duration_ms":15495,"temperature":1.0,"reasoning_tokens":1825,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:14:20.389031+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run P2PNet-Yield on all plots of a new field season with no manual curation and compare the estimated-yield rankings to combine-harvested rankings; if the rank correlation is close to the paper's full-set R2 of 0.0062 rather than the curated R2 of 0.38, the claim that the method ranks genotypes in real breeding fields is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides P2PNet-Soy, the seed-counting model whose backbone is reused as the Feature Extraction Module, plus the external seed-image benchmark used in training."},{"cited_title":"High Throughput Soybean Pod-Counting with In-Field Robotic Data Collection and Machine-Vision Based Data Analysis","cited_arxiv_id":"2105.10568","evidence_quote":"Demonstrates high-throughput soybean pod counting with in-field robotic data collection, establishing the robot-based phenotyping approach."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Reports the strong seed-count-to-yield correlation (r=0.92) that justifies using seed counts as a yield proxy."},{"cited_title":"A., Vasudevan, R., and Johnson-Roberson, M","cited_arxiv_id":null,"evidence_quote":"Supplies the random camera-sensor-effect augmentation method used to improve seed-counting generalization across cameras."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the mvngGrAd moving-grid spatial adjustment, which is applied to ground-truth and estimated values before genotype ranking."}],"review_version":1}