{"id":"2e33ddee-11b0-4785-a544-314d56a2a455","arxiv_id":"2507.05314","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A dual-attention U-Net++ ensemble with class-specific training and Bayesian tuning reports a weighted F1 of 0.8640 on the NBC 2025 wound and scale marker segmentation benchmark.","lead":"Researchers trained separate U-Net++ models for wounds and scale markers, added attention mechanisms, and tuned hyperparameters, reporting a weighted F1 of 0.8640 on the NBC 2025 competition test set. The result is a practical engineering contribution to automated wound assessment, but the architecture combines known components and several supporting claims lack ablations.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 4 states official evaluation was still ongoing yet reports a final externally validated F1 of 0.8640; the central number cannot be trusted until organizers confirm this score belongs to the described pipeline.","rationale":"The reader's weakest_assumption is that the single externally reported weighted F1 of 0.8640 is a valid and correctly attributed measurement of the described submission. My stress-test identifies the same load-bearing concern and sharpens it with a textual inconsistency: Section 4 explicitly says official evaluation was ongoing at manuscript preparation and then reports a final externally validated score. That contradiction makes the central number unverifiable as written. The appropriate disposition remains CONDITIONAL, because the concern is external-validity-based rather than a demonstrated error in the internal experiments. If organizer confirmation is provided, the concern is resolved; if it is not provided, the headline claim should not be accepted as a peer-reviewed finding. I do not find a more load-bearing technical flaw in the architecture or training protocol themselves, though the EfficientNet-B0-versus-B7 discrepancy in the Discussion is a secondary credibility issue. The proposed concrete test is modest and directly targets the disputed quantity, and it avoids relying on hidden ground truth that the authors cannot release.","tokens_in":5789,"tokens_out":2309,"duration_ms":27924,"concrete_test":"Request from the NBC 2025/PCBBE 2025 organizers the official final result for the authors' submission, including submission ID and per-class scores, and verify that the weighted F1 recomputed as 0.75 * F1_wound + 0.25 * F1_marker equals the reported 0.8640. If the organizers confirm this is the final score for the described pipeline, the concern is resolved; if not, the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is the externally validated weighted F1 of 0.8640 in Section 4. The load-bearing premise is that this number is a final, correctly attributed measurement of the described submission. The manuscript itself undermines that premise: Section 4 says \"At the time of manuscript preparation, our models' official evaluation by competition organizers is ongoing\" and then, immediately after, asserts \"Our final externally validated weighted F1-score was 0.8640.\" If official evaluation was still ongoing, a final externally validated score should not yet exist. The paper provides no submission ID, no organizer score report, no date of the evaluation, no per-class external scores, and no protocol for computing the weighted F1 on undisclosed hardware. The abstract and conclusion both use this single decimal as the evidence of effectiveness, so if 0.8640 is stale, preliminary, or computed internally under a different protocol, the headline result collapses even if the internal validation Dice values are accurate. A secondary inconsistency in the Discussion—benchmarking ranked EfficientNet-B0 first, yet the abstract says EfficientNet-B7 was identified as optimal—does not by itself invalidate the chosen pipeline, but it reinforces that the narrative around the final configuration is post hoc and not fully supported by the reported experiments.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript reports a deep-learning pipeline for segmenting wounds and scale markers in clinical images, developed for the NBC 2025 & PCBBE 2025 challenge. The method combines a U-Net++ decoder with a dual-attention module (SCSE and spatial attention), an EfficientNet-B7 encoder, separate models per class, Bayesian hyperparameter optimization (WandB sweeps), test-time augmentation, and optional CRF post-processing. Internal validation Dice are 0.9718 (wounds) and 0.9185 (scale markers), and the paper's headline result is an externally computed weighted F1-score of 0.8640. The manuscript also reports 5-fold benchmarking of multiple architectures and encoders and details the preprocessing and training protocol.","tokens_in":6052,"tokens_out":5669,"duration_ms":60709,"significance":"The paper's potential significance rests on the credibility and generalizability of the external 0.8640 F1-score. If this number is confirmed by the competition organizers and correctly attributed to the described ensemble, the work provides a competent challenge solution and a useful reference configuration for imbalanced medical image segmentation. The external computation by organizers is a genuine strength, as is the systematic 5-fold architecture/encoder benchmarking and the per-class model strategy. However, the current manuscript does not substantiate several load-bearing attributions: the dual-attention module's contribution is never isolated by an ablation, the encoder selection narrative is self-contradictory and omits EfficientNet-B7 from the reported benchmark, and the single externally reported F1 lacks confidence intervals, per-class external scores, and a verification protocol. The scientific novelty is modest because the components are established techniques; the main value is as an engineering benchmark report pending confirmation.","major_comments":[{"comment":"The manuscript states \"At the time of manuscript preparation, our models' official evaluation by competition organizers is ongoing; preliminary internal tests suggest high reliability and robustness\" and then, immediately after, asserts \"Our final externally validated weighted F1-score was 0.8640.\" These two statements are mutually incompatible: a final externally validated score cannot exist while official evaluation is still ongoing. Because the abstract and conclusion both rest on this single number, the central claim is currently unverifiable. Please provide the organizers' score report, submission identifiers, evaluation date, and the exact protocol used to compute the weighted F1, including per-class external scores; without this, the 0.8640 must be treated as preliminary, not as a validated result.","section":"Section 4 (Results)"},{"comment":"There is no ablation isolating the dual-attention module. Section 3.3 introduces the dual-attention enhancement of U-Net++, and the Discussion claims it \"provided significant improvements,\" but Table 2 reports only the final model's Dice. To support this attribution, the paper must compare U-Net++ with and without the SCSE+Spatial module under identical training, hyperparameters, data splits, and ensemble settings, and ideally also ablate TTA, CRF, and the class-specific training strategy. Without such comparisons, the architectural novelty claimed in the title and abstract is not demonstrated.","section":"Sections 3.3, 3.7, and 5 (Discussion)"},{"comment":"The encoder selection is described inconsistently. Section 3.3 says the benchmarking \"identified U-Net++ with EfficientNet-B7 encoder as superior,\" and the abstract repeats that EfficientNet-B7 was \"identified as the optimal encoder backbone.\" However, Section 3.3 lists the benchmarked encoders as DenseNet121, EfficientNet-B0, ResNet34, VGG16, MiT-B0, and MobileNetV2, and Section 5 explicitly states that \"initial benchmarking results (Fig. 1, Fig. 2) demonstrated EfficientNet-B0 as the top-performing encoder\" and that B7 was chosen \"due to its architectural similarity yet greater representational capacity.\" These claims cannot all be true; B7 was apparently never benchmarked in the reported experiments and was therefore not empirically identified as optimal. Please clarify whether B7 was benchmarked, provide the corresponding results, or revise the abstract and Section 3.3 to present B7 as a design choice rather than an empirical finding.","section":"Sections 3.3, 5, and Abstract"}],"minor_comments":[{"comment":"Please clarify how the 163 training images without a second label were used for the scale-marker model: were they excluded, treated as background, or ignored in the loss for that class? This detail affects the reproducibility and the interpretation of the scale-marker Dice.","section":"Section 3.1 (Dataset)"},{"comment":"The text says \"Final performance was averaged across five runs,\" but Table 2 reports four runs. Please reconcile this discrepancy and report the standard deviation or inter-run variability for each class.","section":"Section 3.4 and Table 2"},{"comment":"The ensemble weights w1 and w2, the number of CRF iterations, and whether CRF was used in the final submission are not specified. Please report these values and, if CRF was used, include an ablation showing its effect on the weighted F1.","section":"Appendix A, Algorithm 1"},{"comment":"The Bayesian sweep description is not reproducible: it gives no sweep ranges for the loss ratios, mixup alpha, augmentation strength, or learning rate, no number of trials, and no objective criterion for the sweep. The statement that configurations are \"available in the project's WandB repository\" lacks a URL or access instructions.","section":"Appendix B (Hyperparameter Optimization)"},{"comment":"Figures 1 and 2 need axis labels, metric names, and error bars, and the text should state which validation metric (Dice or IoU) is shown. Currently the reader cannot assess the reported benchmarking ranking from the figures.","section":"Figures 1 and 2"}],"recommendation":"major_revision","confidential_remarks":"This manuscript reads as a competition report rather than a full scientific study. The central result is the externally computed F1, but the self-contradictory statement about ongoing official evaluation and the missing ablation for the dual-attention module mean the claim cannot be accepted as is. The paper would benefit from either organizer confirmation or a clear re-framing as a preliminary technical report. The novelty claim is also somewhat overstated given the existing attention U-Net literature; the contribution is mainly engineering. I would encourage the authors to make the code, sweep logs, and submission verification available before resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a competition write-up, not a methods paper. The headline 0.8640 weighted F1 is an organizer-computed number, which is good, but Section 4 undermines it: “official evaluation by competition organizers is ongoing” is followed one paragraph later by “our final externally validated weighted F1-score was 0.8640.” You cannot have a final external score if the evaluation is still running. No submission ID, score report, date, or protocol is given, so the central result is currently unverifiable. That is a load-bearing flaw, not a cosmetic one.\n\nWhat the paper does well: class-specific training for wound and scale-marker models is sensible for the imbalance here; Bayesian sweeps are appropriate; and the appendix is honest about a training collapse and about the benchmark favoring EfficientNet-B0 before B7 was picked. The internal validation Dice (~0.97/0.92) suggest the pipeline works, and the comparison of many SMP architectures gives a useful empirical snapshot.\n\nSoft spots, in proportion. (1) The external score contradiction above. (2) The Discussion says the benchmark ranked EfficientNet-B0 first while the abstract says B7 was identified as optimal; either the benchmark ranking is not the selection criterion or the wording is post hoc. (3) No ablation isolates the spatial/SCSE attention modules, so “dual-attention helps” is unsupported. (4) The related work already cites a dual-attention U-Net for wound segmentation, so the architectural novelty is modest; the value is in the configuration and the competition result. (5) Code is “available upon reasonable request,” which is not enough for reproducibility.\n\nAll of that said, the paper is not incoherent. The math is simple, there is no fitted-as-predicted issue, and the internal numbers are plausible. The problem is evidentiary, not mathematical.\n\nWho should read it: another team entering NBC/PCBBE, or someone assembling a baseline for wound segmentation. Researchers looking for a new mechanism will learn little.\n\nRecommendation: I would not send this out as-is. If the organizers confirm the score and the authors release code, final hyperparameters, and per-class external scores, a trimmed workshop/competition paper is fine. Until then, desk reject or hold for major revision.","headline":"A single unverified competition F1 is the whole load-bearing wall; the paper itself says official evaluation was still ongoing when it quotes the final number.","tokens_in":6592,"tokens_out":2348,"would_cite":false,"duration_ms":29088,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A dual-attention U-Net++ with an EfficientNet-B7 encoder, class-specific ensembles, and Bayesian hyperparameter tuning achieves a weighted F1 of 0.8640 on the NBC 2025 and PCBBE 2025 wound and scale-marker segmentation benchmark.","keywords":["Image segmentation","Wound assessment","Deep learning","Medical imaging","U-Net++","Attention mechanism","Bayesian optimization","Test time augmentation"],"falsifier":"Ask the competition organizers to release per-class F1 scores for the submitted predictions and recompute the 75/25 weighted combination; if those numbers do not reproduce 0.8640, the headline result is falsified.","tokens_in":5572,"feed_emoji":"🩹","tokens_out":7932,"duration_ms":75401,"temperature":0.7,"pith_summary":"The paper sets out to show that automated wound and scale-marker segmentation can be made accurate and stable enough for clinical use by combining an attention-enhanced U-Net++ with a powerful encoder, per-class training, and careful hyperparameter search. The authors benchmark many architectures and encoders, find U-Net++ with EfficientNet-B7 best, then train separate models for wounds and scale markers with tailored augmentation and Bayesian optimization. They report an externally computed weighted F1 of 0.8640 on the NBC 2025 and PCBBE 2025 challenge test set. If correct, the result demonstrates that a relatively conventional CNN architecture, carefully tuned and ensembled, can handle severely imbalanced clinical segmentation tasks without transformers.","feed_headline":"Attention U-Net++ hits 0.8640 weighted F1 on wound benchmark","feed_subtitle":"Class-specific models and Bayesian tuning make an attention U-Net robust on imbalanced clinical images.","key_machinery":"The load-bearing object is the dual-attention U-Net++: a U-Net++ with nested skip connections, enhanced by a spatial attention module and an SCSE (simultaneous channel and spatial excitation) module in the decoding path, using EfficientNet-B7 as the encoder. Its role is to recalibrate feature maps so that small, irregular wounds and scale markers are emphasized despite severe class imbalance. Class-specific training and test-time augmentation then stabilize the predictions, and the Bayesian sweep tunes loss weights, augmentation strength, learning rate, and mixup alpha for each class-specific model.","core_discovery":"The central claim is that the proposed pipeline—dual-attention U-Net++ (spatial attention plus SCSE channel attention) with an ImageNet-pretrained EfficientNet-B7 encoder, independently trained wound and scale-marker models, Bayesian hyperparameter sweeps, and test-time augmentation—attains a weighted F1-score of 0.8640 on the NBC 2025 and PCBBE 2025 segmentation benchmark. The weighted F1 mixes wound segmentation (75%) and scale-marker segmentation (25%) and was computed by the competition organizers on undisclosed hardware. Internally, the paper reports validation Dice averages of 0.9718 for wounds and 0.9185 for scale markers across four runs. The paper notes that official evaluation was still ongoing when the manuscript was prepared, so the 0.8640 is presented as the final externally validated score that anchors the approach's claimed transfer beyond its own validation split.","pith_inferences":["The authors leave implicit that the gap between the high internal Dice scores and the weighted F1 of 0.8640 could signal test-time distribution shift; per-class external scores would resolve this.","A natural extension would be to ablate the auxiliary pretraining dataset to measure how much of the gain comes from extra data rather than the dual-attention architecture and tuning.","The single externally computed score would carry more weight if paired with a confidence interval across multiple runs, since the paper does not report one."],"forward_implications":["Following the same pipeline—dual-attention U-Net++, EfficientNet-B7, per-class training, and test-time augmentation—should reproduce the reported benchmark performance on similar wound-imaging data.","Training separate models for each class sidesteps the need for complex multi-class loss weighting, since each model sees only one foreground class against background.","The reported internal Dice averages of 0.9718 for wounds and 0.9185 for scale markers give future work concrete baselines to beat on this dataset.","The weighted F1 of 0.8640 sets a reference point for what a carefully tuned CNN ensemble can achieve on the NBC 2025 and PCBBE 2025 challenge."],"supporting_citations":[{"why":"Introduces the encoder-decoder U-Net architecture used as the base in the architecture benchmarking.","marker":"[4]"},{"why":"Presents the dual-attention U-Net approach for wound segmentation that this work extends with a stronger encoder.","marker":"[5]"},{"why":"Introduces the nested skip-connection U-Net++ architecture chosen as the segmentation backbone.","marker":"[7]"},{"why":"Represents one of the segmentation architectures compared during 5-fold cross-validation benchmarking.","marker":"[8]"},{"why":"Provides the auxiliary foot-ulcer wound dataset used for pretraining.","marker":"[11]"}],"fun_headline_variants":["Dual-attention U-Net++ hits 0.8640 on wound benchmark","Class-specific ensembles and Bayesian tuning lift wound F1 to 0.8640","Attention U-Net++ with class-specific models achieves 0.8640 F1","Bayesian-optimized dual-attention U-Net++ scores 0.8640 weighted F1","Wound and scale marker segmentation: F1 0.8640 via attention U-Net++"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the single externally reported weighted F1 of 0.8640 was computed correctly from the described submission; the paper says official evaluation was still ongoing and offers no confidence interval, per-class external scores, or submission protocol.","fun_headline_variants_meta":{"raw":{"variants":["Dual-attention U-Net++ hits 0.8640 on wound benchmark","Class-specific ensembles and Bayesian tuning lift wound F1 to 0.8640","Attention U-Net++ with class-specific models achieves 0.8640 F1","Bayesian-optimized dual-attention U-Net++ scores 0.8640 weighted F1","Wound and scale marker segmentation: F1 0.8640 via attention U-Net++"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000351,"raw_usage":{"total_tokens":1917,"prompt_tokens":949,"completion_tokens":968,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":565,"completion_tokens_details":{"reasoning_tokens":851}},"tokens_in":565,"tokens_out":968,"duration_ms":9622,"temperature":1.0,"reasoning_tokens":851,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:35:08.788384+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Ask the competition organizers to release per-class F1 scores for the submitted predictions and recompute the 75/25 weighted combination; if those numbers do not reproduce 0.8640, the headline result is falsified.","supporting_citations":[{"cited_title":", Baqersad , J","cited_arxiv_id":null,"evidence_quote":"Introduces the encoder-decoder U-Net architecture used as the base in the architecture benchmarking."},{"cited_title":", Ladyzynski , P","cited_arxiv_id":null,"evidence_quote":"Presents the dual-attention U-Net approach for wound segmentation that this work extends with a stronger encoder."},{"cited_title":", Zahia , S","cited_arxiv_id":null,"evidence_quote":"Represents one of the segmentation architectures compared during 5-fold cross-validation benchmarking."}],"review_version":1}