{"id":"69fe5128-637b-4111-b71f-b784acd08b21","arxiv_id":"2411.09283","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Adding an auxiliary patch-level classifier with a class activation map at the bottleneck improves rib fracture segmentation Dice by about 2.3 percentage points over a plain U-Net and about 1.2 points over a FracNet reimplementation.","lead":"This paper adds a small classification network to a standard 3D U-Net to help it find rib fractures in CT scans. The extra branch re-weights the network's internal features and gives a small accuracy boost on the public RibFrac dataset.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's evaluation split is unspecified, and the post-processing thresholds in Table 2 are selected on the same data that later yields the headline Dice/FROC numbers, so the reported 1.846% improvement may be optimistically biased.","rationale":"I checked the architecture and ablation internally: the CAM module is a plausible channel-gating mechanism, the parameter count is consistent with the description, and the ablation comparison is the right one for isolating the auxiliary classifier. I do not see an internal inconsistency that would invalidate the method on its own terms. The weakest point is the evaluation protocol: Table 1 carries no split label, the threshold sweep in Table 2 is performed on what appears to be the same data as the final reported numbers, and no test-set results are shown. This is precisely the condition that must hold for the central quantitative claim to be trusted. The reader's weakest_assumption identifies the same issue, so I agree with the CONDITIONAL verdict. The requested clarification on split identity and threshold selection should be a condition of acceptance; I would not reject the paper outright because the ablation direction is plausible and the reported gains, while possibly optimistic, are not contradicted by internal evidence.","tokens_in":10256,"tokens_out":6687,"duration_ms":66430,"concrete_test":"Ask the authors to specify which split underlies Table 1 and to fix the post-processing thresholds (probability 0.6, size 150, bone 300 HU) using only the RibFrac validation set, then evaluate all final models on the held-out 160-case test split or the official challenge server. Check whether 'Ours(with classifier)' still beats FracNet by a comparable margin; if test Dice differs by more than about 1 point from 64.55, or the relative gain falls below 1.846%, the headline claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the auxiliary classification task with a CAM module improves rib fracture segmentation (Sec. 3.2, Table 1). That claim depends on the quantitative comparison being measured on a split independent of every choice that shaped the model and its post-processing. The paper never states which split Table 1 reports: Sec. 4.2 says only that model selection used minimum validation loss, and the RibFrac 160-case test set is not mentioned again. Table 2 then sweeps probability and size thresholds, and the best row (0.6, 150) has exactly the same scores reported as 'Ours(with classifier)' in Table 1 (FROC 0.71/0.79/0.81/0.81/0.58; DSC 64.55). Thus the final result appears to be the selected maximum of the threshold sweep on the same evaluation split. Because RibFrac test annotations are typically hidden, Table 1 is likely reporting validation-set performance. Threshold selection on the evaluation set inflates absolute FROC/DSC; the 64.55 vs 63.38 (1.846% relative) advantage over FracNet, and the 2.3-point ablation gain over 'without classifier', could shrink or disappear on an unseen split or if baselines receive the same tuned post-processing protocol.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an auxiliary patch-level classification task with a class activation map (CAM) module at the bottleneck of a 3D U-Net to improve rib fracture segmentation on CT patches. The model is trained with a combined segmentation loss (focal + Dice) and binary cross-entropy classification loss, with a schedule that down-weights the classification loss over epochs. Post-processing applies probability, connected-component size, and bone-intensity thresholds. Experiments on the RibFrac dataset compare the method against several baselines (UNet, UNETR, TransUNet, Attention U-Net, FracNet) and report Dice similarity coefficient (DSC) and FROC scores. An ablation with and without the auxiliary classifier shows a 2.34-point DSC improvement and small FROC gains, with only 129 additional parameters.","tokens_in":10488,"tokens_out":4371,"duration_ms":40668,"significance":"If the reported gains are robust, the contribution is practically valuable: a nearly parameter-free auxiliary classifier (129 parameters) that yields consistent segmentation improvements over a strong U-Net baseline, and the CAM-based reweighting at the bottleneck is a simple and general mechanism. The ablation with versus without the classifier directly supports the central claim that the auxiliary task helps. However, the evaluation protocol has a load-bearing weakness: the paper does not state which data split is used for the reported results, and the post-processing thresholds appear to be selected on the same split that produces the headline numbers. This makes the absolute DSC/FROC values and the claimed 1.846% improvement over FracNet potentially optimistically biased. The core idea remains defensible and the issue is fixable, so the manuscript warrants major revision rather than rejection.","major_comments":[{"comment":"The paper never states whether Table 1 reports results on the RibFrac validation split (80 CT scans) or the test split (160 CT scans). Section 4.2 only says model selection used minimum validation loss. More seriously, the 'Ours(with classifier)' row in Table 1 exactly matches the best row of the post-processing threshold sweep in Table 2 (probability 0.6, size 150, FROC 0.71/0.79/0.81/0.81/0.58, DSC 64.55). This indicates that the headline numbers are the selected maximum of a threshold search performed on the same data used for the final evaluation. To support the central claim, the authors must either report results on a held-out test split with thresholds fixed a priori, or explicitly describe a separate validation split used for threshold selection and report the corresponding test-set numbers. Without this, the absolute DSC/FROC values and the 1.846% improvement over FracNet may be optimistically biased.","section":"Sec. 4.2 and Tables 1-2"},{"comment":"The ablation shows FROC improvements at levels 2, 4, and 8 of 0.02 absolute (0.79 vs 0.77, 0.81 vs 0.78, 0.81 vs 0.78), while the reported standard deviations for those metrics are 0.01 for the with-classifier model and up to 0.08 for the without-classifier model. The paper should report whether these differences are statistically significant across the three random initializations (e.g., paired tests or confidence intervals). The DSC improvement of 2.34 points is more compelling, but the FROC gains, which are part of the stated achievements, are marginal and may be within noise.","section":"Sec. 4.4 and Table 1"},{"comment":"The post-processing protocol fixes the bone threshold at 300 HU but sweeps the probability and size thresholds over a small grid, and the best row is reported. This is the same selection-bias issue as in the first comment. Additionally, the FROC values at the 0.5 false-positive level in Table 2 appear internally inconsistent: for probability threshold 0.4, the 0.5-level FROC is 0.81, equal to the level-8 value, while for other thresholds it drops to 0.57-0.59. The authors should verify these numbers and clarify whether the column labeled '0.5' refers to sensitivity at 0.5 false positives per volume or to a different metric.","section":"Sec. 3.3 and Table 2"}],"minor_comments":[{"comment":"Equation (3) uses the notation 𝐺𝑆(Wci) without a formal definition; please clarify that the 're-expansion' operation converts the channel-wise weights back to the spatial dimensions of e3i.","section":"Eq. (3)"},{"comment":"The sentence 'we ensure equal patches for both fracture and non-fracture regions' is ambiguous: it could mean an equal number of patches or equal total volume; please rephrase for clarity.","section":"Sec. 3.1"},{"comment":"The table header lists FROC levels as '1 2 4 8 0.5', which is visually confusing because the 0.5 column appears after the 8 column. Please reorder the columns (e.g., 0.5, 1, 2, 4, 8) and add a footnote defining the FROC metric and the meaning of each column.","section":"Table 1"},{"comment":"The training description does not mention a learning-rate schedule, batch normalization details, or the exact stopping criterion beyond 'minimum validation loss'; these details are needed for reproducibility.","section":"Sec. 4.2"},{"comment":"The CAM visualization is qualitative; consider adding a quantitative measure (e.g., IoU between a thresholded CAM and the ground-truth fracture mask) to support the claim that CAM highlights fracture-related regions.","section":"Fig. 5"},{"comment":"References [13] and [30] cite the same RibFrac challenge paper; please merge them to avoid duplication.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is a short conference paper (ICVGIP 2024) and the central idea is sound, but the evaluation protocol issue is significant: the unspecified split and the threshold selection on the evaluation set undermine the quantitative claims. This is fixable by re-running the evaluation on a held-out test split or by clearly separating validation-based threshold selection from test reporting. I would also encourage the authors to make the code and data splits available. The small FROC differences in the ablation should be addressed with statistical tests. Overall, the manuscript merits a major revision, not rejection, because the ablation design provides direct evidence for the contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nShort version: the paper does what it says on the tin – adds a patch-level auxiliary classifier with CAM re-weighting to a U-Net and reports better rib fracture segmentation on RibFrac – but the quantitative evidence is weaker than the abstract implies. The ablation comparing with and without the classifier is the core evidence, and it is the right kind of experiment. The gain is real in their setup: about 2.3 Dice points and a few FROC points at levels 2, 4, and 8. I think that part is believable.\n\nWhat is actually new is the specific combination of an auxiliary classification head at the bottleneck, using the classifier's CAM to re-weight bottleneck features, applied to rib fracture segmentation. Each component is standard, but I don't know of another paper doing exactly this on RibFrac. That is a modest contribution, not a breakthrough.\n\nThe soft spot is the evaluation protocol. The paper never states which split Table 1 reports. Section 4.2 says model selection used minimum validation loss, and the test set is never mentioned again. Table 2 sweeps probability and size thresholds, and the best row on that sweep (0.6, 150) matches the exact numbers reported for 'Ours(with classifier)' in Table 1. That is a red flag: the headline numbers appear to be the result of threshold tuning on the same data used for the final evaluation. Since RibFrac test annotations are typically hidden, Table 1 is almost certainly validation-set performance. Threshold selection on the validation set inflates absolute scores, and the relative advantage over baselines could shrink or disappear if baselines got the same tuning. The missing hyperparameters (alpha1, alpha2, gamma, theta schedule) and no released code make it hard to reproduce or judge robustness.\n\nI'm not saying the method doesn't help. The ablation is internally consistent, and the visual examples are suggestive. But the quantitative claim of 'significant improvement' should be taken with salt until the authors clarify the split and show that thresholds were chosen on a separate validation partition.\n\nWho is this for? People working on rib fracture segmentation or weakly supervised approaches to medical image segmentation. They would read the method section and maybe the ablation, but they should not quote the absolute numbers without checking the protocol.\n\nRecommendation: send it to peer review with a request for clarification on the evaluation protocol. If the authors can show the thresholds were tuned on a separate split and report test-set numbers, the result would be credible. If not, the paper should be revised to stop presenting tuned validation numbers as a fair comparison.","headline":"Plausible incremental improvement whose headline numbers are compromised by post-processing thresholds tuned on the evaluation set.","tokens_in":11040,"tokens_out":2152,"would_cite":false,"duration_ms":71090,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding an auxiliary patch-level classifier with a class activation map at the bottleneck of a 3D U-Net improves rib fracture segmentation on CT by about 1.8% Dice over published baselines.","keywords":["rib fracture segmentation","class activation map","auxiliary classification","weak supervision","3D U-Net","computed tomography","RibFrac","FROC"],"falsifier":"Re-run the experiment with thresholds fixed by cross-validation on the training set only, then evaluate on the original test split; if the with-classifier model no longer beats the no-classifier model by roughly 2.3 Dice points and 2.5-2.9% FROC at levels 2, 4, and 8 (i.e., the gap falls within one standard deviation of the three random initializations), the reported improvement is a threshold-selection artifact rather than a property of the auxiliary classifier.","tokens_in":10011,"feed_emoji":"🩻","tokens_out":8786,"duration_ms":78354,"temperature":0.7,"pith_summary":"Rib fractures are hard to segment automatically because they appear in many sizes and shapes across dozens of CT slices. The paper claims that adding a patch-level \"does this patch contain a fracture?\" classification task, with a class activation map (CAM) module at the bottleneck of a 3D U-Net, makes the bottleneck features highlight fracture-relevant regions and thereby improves segmentation. On the RibFrac dataset the model reports an overall Dice gain of 1.846% over comparison baselines, with FROC sensitivity gains at 2, 4, and 8 false positives per volume and a Dice improvement of about 2.3% over the same U-Net without the classifier, at a cost of only 129 extra parameters. If the result holds, it offers a cheap way to inject weak supervision into medical-image segmentation without acquiring more voxel-level labels.","feed_headline":"Patch-level classifier lifts rib fracture Dice by 1.8%","feed_subtitle":"A 129-parameter CAM module at the U-Net bottleneck beats heavier baselines on RibFrac CT scans.","key_machinery":"The central object is the CAM module at the bottleneck: global average pooling compresses the encoder output to a vector, a linear layer predicts fracture presence, and the same linear-layer weights are re-expanded and passed through a sigmoid to form a per-channel attention map that is multiplied back into the encoder features before the decoder. This re-weighting focuses the decoder on fracture-like regions, while the epoch-decreasing classification-loss weight theta(tau) lets the auxiliary task guide early learning without dominating later segmentation.","core_discovery":"The paper's central claim is that an auxiliary classification head at the bottleneck, implemented as a global-average-pooled linear layer whose weights are re-expanded and applied as a channel-wise attention mask through a sigmoid, is what carries the segmentation improvement. The classifier is trained jointly with the segmentation loss (focal plus Dice) using binary cross-entropy on patch labels; the classification loss is weighted by a monotonically decreasing function of epoch so early classification errors do not destabilize later segmentation. In the authors' experiments on RibFrac, the model with this module reaches 64.55% Dice and FROC sensitivities of 0.58, 0.71, 0.79, 0.81, and 0.81 at false-positive levels 0.5, 1, 2, 4, and 8, compared with 62.21% Dice and 0.60, 0.71, 0.75, 0.78, and 0.78 for the same U-Net without the classifier, and it outperforms UNETR, TransUNet, Attention U-Net, and FracNet under matched preprocessing and postprocessing.","pith_inferences":["The same bottleneck classifier could be dropped into other small-lesion segmentation tasks (vertebral fractures, pulmonary nodules, micro-cracks in materials CT) where patch-level positive/negative labels are easier to collect than voxel masks, though the paper only demonstrates rib fractures.","The CAM maps shown in the paper are a by-product of the mechanism; a quantitative evaluation of their localization accuracy (for example, lesion-centric Dice between CAM and ground truth) would tell whether the classifier is genuinely highlighting fracture regions or merely correlated features.","Because the no-classifier model wins at FROC level 0.5, the auxiliary task appears to trade away some high-sensitivity operating points; a testable extension is to fuse the two heads at inference or tune the classification-weight schedule to recover that sensitivity without losing the gains at levels 2, 4, and 8.","The post-processing has three thresholds (probability, size, and bone intensity) that are tuned jointly in the paper, so an ablation isolating each threshold would show how much of the reported gain over baselines comes from the CAM module versus optimal threshold selection."],"forward_implications":["Adding the auxiliary classifier raises Dice from 62.21% to 64.55% on the same U-Net backbone, so the gain is attributable to the auxiliary task rather than to model capacity.","FROC sensitivity improves by 2.5%, 2.9%, and 2.8% at false-positive levels 2, 4, and 8, meaning fewer fractures are missed at operating points relevant for screening.","The with-classifier model uses only 129 additional parameters yet matches or exceeds much larger baselines such as UNETR, TransUNet, and Attention U-Net under matched preprocessing and postprocessing.","The epoch-decreasing classification weight lets the model use patch labels early in training and then concentrate on the voxel segmentation objective, which is why the extra task does not degrade final performance.","Overall, the model reports a 1.846% Dice improvement over the baseline models compared in Table 1."],"supporting_citations":[{"why":"Provides the base U-Net architecture that the proposed model extends with the CAM-based auxiliary classifier.","marker":"[22]"},{"why":"Provides the FracNet baseline and the connected-component/filtering post-processing approach that the paper adapts for fair comparison.","marker":"[12]"},{"why":"Supplies the RibFrac dataset of 660 CT volumes with rib fracture annotations used for training, validation, and testing.","marker":"[13]"},{"why":"Supplies the class activation map mechanism, re-expanding linear classifier weights to highlight discriminative regions, which the CAM module implements.","marker":"[32]"},{"why":"Provides the precedent of an auxiliary bottleneck task improving segmentation, motivating the auxiliary classifier.","marker":"[18]"},{"why":"Provides the precedent of coupling a segmentation subtask with classification in a weakly supervised setting, which the paper builds on.","marker":"[21]"},{"why":"Is a comparison baseline (UNETR transformer encoder) evaluated under the same preprocessing and postprocessing.","marker":"[9]"},{"why":"Is a comparison baseline (TransUNet) evaluated under the same preprocessing and postprocessing.","marker":"[5]"},{"why":"Is a comparison baseline (Attention U-Net) evaluated under the same preprocessing and postprocessing.","marker":"[20]"}],"fun_headline_variants":["129-parameter sigmoid mask from classifier lifts rib fracture Dice by 2.3 points","Auxiliary head's sigmoid mask yields 2.3-point Dice gain over plain U-Net","On RibFrac, classifier-head attention lifts Dice 2.3 pts","Channel-wise attention from classifier head ups rib fracture Dice by 2.3 pts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported advantage over baselines rests on the assumption that the post-processing thresholds (probability 0.6, size 150, bone 300 HU) and the model checkpoint were selected using only validation data, and that the numbers in Table 1 are from a held-out set the authors never tuned on; the paper does not explicitly document which split produced Table 1.","fun_headline_variants_meta":{"raw":{"variants":["129-parameter sigmoid mask from classifier lifts rib fracture Dice by 2.3 points","Auxiliary head's sigmoid mask yields 2.3-point Dice gain over plain U-Net","On RibFrac, classifier-head attention lifts Dice 2.3 pts","Channel-wise attention from classifier head ups rib fracture Dice by 2.3 pts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001595,"raw_usage":{"total_tokens":6347,"prompt_tokens":923,"completion_tokens":5424,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":539,"completion_tokens_details":{"reasoning_tokens":5340}},"tokens_in":539,"tokens_out":5424,"duration_ms":39888,"temperature":1.0,"reasoning_tokens":5340,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:48:35.842881+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the experiment with thresholds fixed by cross-validation on the training set only, then evaluate on the original test split; if the with-classifier model no longer beats the no-classifier model by roughly 2.3 Dice points and 2.5-2.9% FROC at levels 2, 4, and 8 (i.e., the gap falls within one standard deviation of the three random initializations), the reported improvement is a threshold-selection artifact rather than a property of the auxiliary classifier.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the FracNet baseline and the connected-component/filtering post-processing approach that the paper adapts for fair comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the RibFrac dataset of 660 CT volumes with rib fracture annotations used for training, validation, and testing."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the precedent of an auxiliary bottleneck task improving segmentation, motivating the auxiliary classifier."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the precedent of coupling a segmentation subtask with classification in a weakly supervised setting, which the paper builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is a comparison baseline (UNETR transformer encoder) evaluated under the same preprocessing and postprocessing."}],"review_version":1}