{"id":"3d38459d-de7f-410a-8efe-caaf53e77d62","arxiv_id":"2411.11738","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A customized YOLO detector, WoodYOLO, reports F2 0.848 at IoU 0.3 for vessel-element detection in wood microscopy, beating YOLOv10 and YOLOv7 on a private dataset.","lead":"WoodYOLO adapts the YOLO object detector to find vessel cells in microscope images of wood pulp, reporting gains of 12.9% and 6.5% in F2 score over YOLOv10 and YOLOv7. The work targets automated wood species verification for EU deforestation regulation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed significant outperformance rests on a single 613/154 split and a custom F2@IoU 0.3 metric, with no reported variance or confidence intervals; the paper mentions 5-fold CV stability but does not report those results.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the evaluation protocol. The paper's own text states 'Due to time constraints, we use a simple train-validation split' and mentions 'initial experiments with 5-fold cross-validations' without reporting fold-wise metrics. My stress-test adds the specific mechanism of threshold selection on the custom F2@0.3 metric, which can further inflate the gap. This does not change the reader's conditional verdict: the architectural work and ablations are plausible and reasonably detailed, but the headline claim of significant outperformance requires variance-aware evaluation or released artifacts to be confirmed. I recommend no verdict change beyond the reader's CONDITIONAL, since the necessary evidence is missing rather than contradicted.","tokens_in":12102,"tokens_out":3395,"duration_ms":32540,"concrete_test":"Run 5-fold cross-validation on the 767 annotated images using the exact WoodYOLO configuration and at least YOLOv7-W6, YOLOv7-tiny, and YOLOv10-M under identical training schedules and image resolutions. Report per-fold F2@0.3 with mean±std and a 95% bootstrap confidence interval over validation images. Also perform a confidence-threshold sweep on a held-out fold to verify that the 0.848 result is not an artifact of selecting the threshold on the validation split. If the confidence intervals for WoodYOLO and YOLOv7-W6 overlap, or the gap is less than one standard deviation, the claimed 6.5% gain is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract, §4) is that WoodYOLO significantly outperforms YOLOv10 and YOLOv7 by 12.9% and 6.5% in F2 score. The load-bearing assumption is that the single train/validation split (613/154 images, §4) is representative enough to make a 6.5-point gap meaningful. The paper provides no variance estimate: no per-fold or per-image error bars, no bootstrap confidence intervals, and the 5-fold cross-validation mentioned in §4 is described only as 'relatively stable' without reported numbers. Because the metric is a custom F2 at a single IoU threshold of 0.3 (§3.3), precision/recall depend on the confidence threshold; if that threshold is selected on the same 154-image validation set, the reported gap can be inflated by threshold overfitting. A 154-image validation set also has high sampling variability, especially since image-level vessel counts vary widely, so the F2 estimate may be dominated by a few images. Without a measure of spread, the word 'significantly' is not supported. The missing YOLOv8 baseline is secondary; the primary issue is that the evaluation protocol cannot currently rule out that the 6.5% advantage over YOLOv7-W6 is within split-to-split noise.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces WoodYOLO, a YOLO-based object detector for locating vessel elements in large, high-resolution microscopic images of macerated hardwood. The authors adapt a YOLOv7-tiny-style architecture with a VGG11 backbone, a custom box parametrization using two scalar hyperparameters (maximum width and height) instead of anchor boxes, a two-term loss combining IoU regression and BCE confidence, and a task-specific F2 metric at IoU 0.3. On a dataset of 767 images (613 train / 154 validation) they report an F2 score of 0.848 for WoodYOLO, outperforming YOLOv10-S/M and YOLOv7-tiny/W6 by 12.9% and 6.5%, respectively, while using less VRAM. The paper also presents ablations of backbones, multi-positive neighbor settings, IoU loss variants, image size, and training techniques such as mosaic augmentation.","tokens_in":12366,"tokens_out":3046,"duration_ms":29885,"significance":"If the reported gains are reproducible, the work is a useful domain application: it demonstrates that a custom, lightweight YOLO variant can beat general-purpose detectors on a specialized high-resolution microscopy task, and it documents that common COCO-oriented techniques (mosaic augmentation, multi-positives, decoupled heads) do not transfer to this setting. The authors provide a substantial amount of experimentation and a from-scratch implementation, which is a strength for reproducibility, though no code or data is released. The scientific significance is currently limited by the evaluation protocol: the central claim of 'significant' improvement rests on a single split without error bars, a custom metric at a single IoU threshold, and comparisons at different input resolutions. These issues are fixable and the contribution is potentially publishable, but the evidence as presented does not yet support the strength of the claim.","major_comments":[{"comment":"The central claim that WoodYOLO 'significantly outperforms' YOLOv10 and YOLOv7 is not supported by statistical evidence. The evaluation uses a single train/validation split (613/154 images), and although §4 states that 'initial experiments with 5-fold cross-validations ... found that the metrics are relatively stable across different folds', no fold-wise numbers, means, standard deviations, confidence intervals, or per-image variance are reported. With 154 validation images and wide variability in vessel counts per image, the 6.5% F2 gap over YOLOv7-W6 could plausibly be within split-to-split noise. Please report the 5-fold results (or a bootstrap over images) with mean ± std for all compared models, and avoid the word 'significant' unless a significance test or non-overlapping intervals are provided.","section":"§4, Table 1 and abstract"},{"comment":"The custom F2 metric computed at a fixed IoU threshold of 0.3 is threshold-dependent, and the paper does not state how the confidence threshold is chosen for each model. If the confidence threshold (or any hyperparameter) is selected on the same 154-image validation set, the reported gap can be inflated by threshold overfitting. Please report precision-recall curves, the F2 value at multiple confidence thresholds, and additionally provide a standard metric such as AP or mAP at IoU 0.5 and 0.5:0.95 to show that the improvement is not an artifact of the specific low IoU threshold. Also state explicitly how the confidence threshold is set for each model in Table 1.","section":"§3.3 and §4"},{"comment":"The comparison is not apples-to-apples because models are evaluated at different input resolutions: YOLOv7-W6 is run at 5184×5184 while WoodYOLO uses 2048×2048, and the text explains the higher resolution requires an A100. Higher input resolution generally improves detection performance but with a compute cost, so the reported F2 gaps conflate architectural differences with resolution and resource differences. Please either run the baselines at the same resolution as WoodYOLO, or report performance as a function of resolution and VRAM / inference time for all models. Additionally, YOLOv8 is mentioned in the related work (Qamar et al. use it for macerated fiber segmentation) but is omitted from Table 1; since YOLOv8 is a current state-of-the-art baseline, it should be included or its exclusion justified.","section":"§4.1, Table 1"},{"comment":"Several ablation conclusions are based on differences that are small relative to the rounding shown, and no variance estimates are provided. For example, Table 4 shows IoU loss variants spanning only 0.8293–0.8340, yet the text concludes 'GIoU yielded the best performance'; Table 5 shows image size 2048 giving 0.8316 versus 4964/4096 giving 0.8243, and Table 6 reports the anchor-box constraint contribution as 0.848 vs 0.841 (0.7%). Without repeated runs or error bars, these differences may be noise. Please provide uncertainty estimates for the ablations or explicitly state which differences are below the resolution of the evaluation.","section":"§4.2, §4.3, Tables 3–6"}],"minor_comments":[{"comment":"The notation in the loss equations is confusing: 'm' is used for the number of bounding boxes in the loss while the same letter is used for the hyperparameter 'm' in the box parametrization; also 'n' and 'm' are not clearly tied to the grid dimensions. Please rename or define all symbols explicitly.","section":"§3.2, Eqs. (2)–(3)"},{"comment":"The box parametrization w = σ(f)^2 · gwi · mw and h = σ(f)^2 · ghi · mh is ambiguous: the text says mw and mh define the maximum width/height relative to the image, but the formula multiplies by the grid width gwi and grid height ghi. It would be clearer to present the scaling to input-image coordinates explicitly, including how the grid indices are added for xc and yc.","section":"§3.1, Eq. (1)"},{"comment":"The statement 'YOLOv5 and YOLOv8 have never been published' is imprecise and may distract readers; while those versions lack peer-reviewed papers, they are widely used and the related work itself cites a YOLOv8-based study. Rephrase to say they have not been formally peer-reviewed or provide a citation for that claim.","section":"§2"},{"comment":"The sentence 'The parameters of YOLOv10 and YOLOv7 have both been optimized' is unclear — does it mean hyperparameters, pretrained weights, or inference settings? Please specify what optimization was performed for each baseline.","section":"§4.1"},{"comment":"No code or data availability statement is provided. Given the emphasis on reproducibility (the authors note undocumented workarounds in other YOLO implementations), a public release of code and, where possible, annotations would strengthen the paper.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a real application and contains a substantial amount of empirical work, but the evaluation protocol is the main blocker. If the authors can provide cross-validation or bootstrap error bars, add standard metrics, match resolutions, and include YOLOv8, the contribution could be accepted. I also note that the paper's framing as 'novel' is modest — the architecture is a lightweight YOLOv7-tiny variant with a constrained box parametrization — but that is acceptable for a domain-specific detector if the empirical claims hold."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The takeaway: this is a solid engineering paper for a niche but real problem — detecting vessel elements in microscopic wood images for timber regulation compliance. The headline claim is a 12.9% (absolute) F2 gain over YOLOv10 and 6.5% over YOLOv7 at IoU 0.3. The gains are plausible given the magnitudes, but the evaluation is not rigorous enough to support the word \"significantly.\"\n\nWhat's actually new: the max-width/height anchor scheme (mw, mh) is a simple, sensible simplification of anchor boxes, and the ablations are useful. The negative results — mosaic augmentation hurts, multi-positives don't help, decoupled head doesn't help — are the kind of domain-specific findings that general-purpose YOLO papers rarely report. The paper is honest: it admits the 5-fold CV was abandoned due to time and notes that YOLOv5/v8 were never formally published. Building the detector from scratch to avoid legacy code workarounds is good practice. The citation pattern is fine — they build on their own prior work and the relevant YOLO literature.\n\nSoft spots, in order of severity. The single 613/154 split is the big one. The paper mentions that 5-fold CV was \"relatively stable\" but gives no numbers. Without variance, the 6.5-point gap over YOLOv7-W6 could be split noise. The custom F2@0.3 metric is defensible for the application, but if the confidence threshold was tuned on the same 154 validation images, the gap can be inflated. They need per-fold results or bootstrapped confidence intervals, or at least to tone down \"significantly.\"\n\nSecond, baseline comparisons use different input resolutions: YOLOv7-W6 gets 5184x5184, WoodYOLO gets 2048x2048. That is actually a point in their favor — less resolution, less VRAM, higher F2 — but it is not a clean control. A YOLOv7-W6 run at 2048 would strengthen the comparison. Third, the missing YOLOv8 baseline is minor but annoying, since they cite a YOLOv8 paper for the same task. Fourth, the presentation is slightly inconsistent: backbone and loss ablations use 2 neighbors, while the final config uses 0 neighbors; it is clear from the text but easy to misread. No code or data release is a reproducibility gap.\n\nOverall, the central claim likely holds — WoodYOLO is a good detector for this task — but \"significantly outperforms\" is not supported as written. With variance estimates, this would be a strong niche contribution. It deserves a serious referee; the review should ask for error bars, a YOLOv8 baseline, and ideally code release. I see this as major revision rather than rejection.","headline":"Solid niche engineering paper with a plausible but statistically unsupported headline claim; the single-split evaluation with no error bars is the main weakness.","tokens_in":12904,"tokens_out":3588,"would_cite":false,"duration_ms":32741,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"WoodYOLO, a purpose-built YOLO variant for microscopic wood images, reaches an F2 score of 0.848 at IoU 0.3 and beats YOLOv7-W6 by 6.5 points and YOLOv10-M by 12.9 points.","keywords":["object detection","microscopic imaging","wood species identification","vessel element detection","YOLO","high-resolution microscopy","F2 score","forest protection"],"falsifier":"Train WoodYOLO and YOLOv7-W6 on the same 613 images and evaluate on five random 154-image validation folds, reporting mean and standard deviation of F2 at IoU 0.3 and at IoU 0.5, plus COCO-style average precision; if the mean gap shrinks below about two points or the ordering flips at the stricter threshold, the claim of significant outperformance is not supported.","tokens_in":11929,"feed_emoji":"🔬","tokens_out":9739,"duration_ms":79164,"temperature":0.7,"pith_summary":"The paper sets out to automate a bottleneck in wood species verification: locating vessel elements, the water-conducting cells whose shapes differ among hardwood genera, in very large microscope images of macerated wood. It presents WoodYOLO, a custom YOLO-style detector trained at 2048-by-2048 resolution with a single-class head, no anchor-box clustering, and a pair of hyperparameters that cap object width and height. On a dataset of 767 images with 118,287 annotated vessel elements, the best configuration scores F2 0.848 at an IoU threshold of 0.3, compared with 0.783 for YOLOv7-W6 and 0.719 for YOLOv10-M, while using roughly 3-4 times less memory. The paper's broader claim is that general-purpose detector designs tuned on COCO do not automatically transfer to narrow, high-resolution domains, and that a from-scratch task-specific detector can win instead.","feed_headline":"Custom YOLO beats YOLOv7 and YOLOv10 on wood-fiber images","feed_subtitle":"A purpose-built detector gains 6.5-12.9 points in F2 while using a fraction of the memory.","key_machinery":"The load-bearing mechanism is a single-class, YOLO-style detection head that outputs five values per grid cell — center x, center y, width, height, and confidence — with width and height formulas $\\sigma(f)^2 g_w m_w$ and $\\sigma(f)^2 g_h m_h$. The hyperparameters $m_w, m_h \\in [0,1]$ fix the maximum object size, replacing the usual anchor-box clustering step and simplifying the loss. Training combines an IoU-based regression loss with a binary cross-entropy confidence loss, and the evaluation metric is F2 at a fixed IoU threshold of 0.3, chosen because the application needs high recall and tolerates imprecise box overlap.","core_discovery":"The central claim is that a purpose-built detector can beat state-of-the-art general YOLO variants on the specific task of vessel-element localization in microscopic wood images. WoodYOLO combines a shallow YOLOv7-tiny-derived neck with a single-class prediction head and a size-capping scheme: predicted width and height are written as $\\sigma(f)^2 g_w m_w$ and $\\sigma(f)^2 g_h m_h$, where $m_w$ and $m_h$ are user-set maximum fractions of image width and height. With this configuration the paper reports F2 0.848 at IoU 0.3, outperforming YOLOv7-W6 (0.783) and YOLOv10-M (0.719) and matching or beating the need for a 5184-by-5184 input with a much lighter 2048-by-2048 training resolution. The paper also reports that mosaic augmentation lowers F2 by 6.2 points, multi-positive center sampling helps only with zero neighbors, and other COCO-oriented tricks do not help, supporting its argument for task-specific design.","pith_inferences":["If the reported margins were re-evaluated with 5-fold cross-validation and standard AP at IoU 0.5, the ranking might shift; the paper's own admission that the split was chosen for time constraints makes this a natural next check.","The F2@0.3 metric is so recall-favoring that part of the advantage over YOLOv7-YOLOv10 may come from the metric rather than from localization quality; a follow-up that filters false positives with a classifier, which the authors already propose, would show how much of the margin is real.","The size-capping head is a transferable idea: any single-class detection task with a known maximum object size (cell counting, particle detection, aerial imagery) could adopt the same trick without needing anchor-box clustering.","The broader lesson that COCO optimizations fail to transfer is drawn from one dataset; replicating the ablation on a second microscopy or satellite dataset would test whether it is a general phenomenon or a property of this particular image type."],"forward_implications":["Automated vessel-element localization of this quality could make paper and pulp wood-species verification faster, cheaper, and less dependent on a small pool of expert wood anatomists.","For domains far from general-purpose benchmarks, a custom detector built from scratch can outperform newer general YOLO releases, meaning version number is not a reliable proxy for real-world performance.","COCO-proven techniques such as mosaic augmentation can actively hurt a specialized detector; the paper's ablations give a concrete counterexample with a 6.2-point F2 drop.","The size-capping anchor scheme removes the need for anchor clustering and should simplify deployment on other single-class detection tasks with bounded object sizes.","WoodYOLO's 8-10 GB training memory at 2048-by-2048 makes high-resolution microscopy detection feasible on consumer-grade hardware, and the architecture scales up to 6144-by-6144 when more memory is available."],"supporting_citations":[{"why":"Provides the prior two-step detection-and-classification pipeline and the macerated-wood dataset that WoodYOLO extends and improves.","marker":"[Nieradzik et al., 2023]"},{"why":"Supplies the YOLOv7 baseline (W6 and tiny) and the neck architecture WoodYOLO adapts for its detector.","marker":"[Wang et al., 2022]"},{"why":"Supplies the YOLOv10 baseline that WoodYOLO outperforms, the main state-of-the-art comparison.","marker":"[Wang et al., 2024b]"},{"why":"Source of the multi-positives and anchor-free design choices whose transfer the paper tests and finds unhelpful.","marker":"[Ge et al., 2021a]"},{"why":"Source of mosaic augmentation, which the paper ablates and finds to reduce F2 by 6.2 points.","marker":"[Bochkovskiy et al., 2020]"},{"why":"Defines the AP metric the paper argues is unsuitable for recall-critical localization, motivating the F2@0.3 metric.","marker":"[Everingham et al., 2010]"},{"why":"Describes the maceration method used to prepare the wood samples into individual cells for imaging.","marker":"[Franklin, 1945]"},{"why":"Provides the anatomical atlas used to identify and annotate vessel elements in the reference material.","marker":"[Helmling et al., 2018]"}],"fun_headline_variants":["WoodYOLO beats YOLOv7 and YOLOv10 on wood-fiber images","WoodYOLO gains 6.5–12.9 F2 over YOLO with 1/4 resolution","WoodYOLO: 2048px beats 5184px for wood-cell localization","Skipping mosaic lifts WoodYOLO's F2 by 6.2 points"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison leans entirely on a single 613/154 train-validation split and a custom F2 metric at IoU 0.3; if that split is unrepresentative or the threshold is too forgiving, the reported margin over YOLOv7 and YOLOv10 may not generalize.","fun_headline_variants_meta":{"raw":{"variants":["WoodYOLO beats YOLOv7 and YOLOv10 on wood-fiber images","WoodYOLO gains 6.5–12.9 F2 over YOLO with 1/4 resolution","WoodYOLO: 2048px beats 5184px for wood-cell localization","Skipping mosaic lifts WoodYOLO's F2 by 6.2 points"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001197,"raw_usage":{"total_tokens":4921,"prompt_tokens":916,"completion_tokens":4005,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":532,"completion_tokens_details":{"reasoning_tokens":3902}},"tokens_in":532,"tokens_out":4005,"duration_ms":28709,"temperature":1.0,"reasoning_tokens":3902,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:11:16.586846+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train WoodYOLO and YOLOv7-W6 on the same 613 images and evaluate on five random 154-image validation folds, reporting mean and standard deviation of F2 at IoU 0.3 and at IoU 0.5, plus COCO-style average precision; if the mean gap shrinks below about two points or the ordering flips at the stricter threshold, the claim of significant outperformance is not supported.","supporting_citations":[{"cited_title":"Automating wood species detection and classification in microscopic images of fibrous materials with deep learning, 2023","cited_arxiv_id":null,"evidence_quote":"Provides the prior two-step detection-and-classification pipeline and the macerated-wood dataset that WoodYOLO extends and improves."},{"cited_title":"Preparation of thin sections of synthetic resins and wood-resin composites, and a new macerating method for wood","cited_arxiv_id":null,"evidence_quote":"Describes the maceration method used to prepare the wood samples into individual cells for imaging."},{"cited_title":"Atlas of vessel elements: Identification of asian timbers","cited_arxiv_id":null,"evidence_quote":"Provides the anatomical atlas used to identify and annotate vessel elements in the reference material."}],"review_version":1}