{"id":"2117259c-1edf-48d5-8cbe-1f08c778ac4d","arxiv_id":"2504.12203","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A denoising autoencoder trained on clean organ masks detects inaccurate auto-segmentations from any modality, using reconstruction Dice loss as the inaccuracy score.","lead":"The authors train a 3D denoising autoencoder on clean organ masks; at test time, the Dice loss between an auto-segmentation and its reconstruction flags inaccurate segmentations. The method is tested on MR pelvis and CT kidney cases and compares favorably with VAE and statistical baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Training noise model only covers patch-like errors, so the DAE may miss systematic auto-segmentation errors; the general detection claim is not yet supported.","rationale":"The reader's conditional verdict identifies the training noise model as the weakest assumption, and this stress-test agrees. The DAE's notion of inaccuracy is defined entirely by the random-patch corruption used in training; the inaccuracy score is only meaningful for errors that the DAE has learned to denoise. The paper demonstrates strong AUROC on two use cases, but the errors shown in Figure 4 and described in Section 3 are patch-like (missing tissue, disconnection). Systematic errors such as global boundary shifts or organ confusion are never tested, so the central claim that the method detects inaccurate organ segmentations across modalities and segmentation models is broader than the evidence. The concern is not that the method is wrong, but that its validity outside the patch-noise regime is unestablished. A synthetic perturbation test would directly settle whether the DAE flags systematic errors. Since this is the same assumption the reader flagged, the conditional verdict remains appropriate; no new fatal flaw was found.","tokens_in":6886,"tokens_out":4662,"duration_ms":55044,"concrete_test":"Using the same MR-pelvis DAE and preprocessing, feed ground-truth masks synthetically corrupted by (a) morphological erosion and dilation by 1 to 10 mm, (b) smooth sinusoidal boundary displacement, and (c) organ-label confusion. Compute the inaccuracy score versus the magnitude of each error type and compare AUROC for detecting these systematic errors against the existing patch-corruption results. If AUROC is near 0.5 or the score does not correlate with Dice-to-ground-truth for these error types, the method fails on errors outside the patch-noise class, undermining the general claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism is trained exclusively on ground-truth masks corrupted by adding or removing random binary patches (Section 2.4). The inaccuracy score is then defined as the Dice loss between the auto-segmentation and the DAE reconstruction. For the method to detect inaccurate segmentations in general, this score must respond to the full range of clinically relevant auto-segmentation errors. The paper's two validations, however, exhibit largely patch-like failure modes: missing coverage at the femoral head and a disconnected kidney due to erosion (Figure 4, Section 3). No evidence is provided that systematic errors—global boundary contraction or expansion, smooth boundary shifts, or organ confusion—produce high Dice loss. Because the DAE is trained to undo only the injected patch noise, it may treat a smoothly deformed but still plausible mask as a valid clean shape and reconstruct it with low loss, yielding a low inaccuracy score. Thus the claimed generality to arbitrary auto-segmentation errors is an internal gap between the training noise distribution and the scope of the claim, not merely a disagreement with external consensus.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an explainable, modality-independent method for detecting inaccurate organ segmentations in radiation therapy planning. A denoising autoencoder (DAE) is trained on ground-truth organ masks that are corrupted by adding or removing random binary patches; at inference, the inaccuracy score is the Dice loss between an auto-segmentation and the DAE's reconstruction. The method is evaluated on MR pelvis auto-segmentations (seven organs) and CT kidney auto-segmentations (two organs), and compared against a VAE-based approach following Sandfort et al. and a statistical Mahalanobis-distance baseline. The authors report superior performance of the DAE for all seven MR pelvis organs and comparable performance for the CT kidneys, and claim modality independence because only binary masks are used.","tokens_in":7074,"tokens_out":3043,"duration_ms":31378,"significance":"If the central claim holds, the method is a practical post-hoc quality-assurance tool for organ-at-risk segmentation: it does not require access to the segmentation model, uses only binary masks, and provides visual explanations via reconstructions. The idea is simple, plausible, and could be useful in clinical workflows where automated segmentation is increasingly used. The authors are explicit about the training setup and evaluation metrics, and the inclusion of bootstrap confidence intervals is commendable. However, the evidence for the generality of the method is incomplete: the training noise model covers only a narrow class of segmentation errors, the statistical baseline is implemented with a reduced feature set, and the CT kidney results are only comparable rather than superior. These gaps do not invalidate the approach but do limit the strength of the claims one can currently make.","major_comments":[{"comment":"The training noise model injects or removes random binary patches from ground-truth masks, and the method is then claimed to detect inaccurate organ segmentations in general. However, the two validation examples in Figure 4 (insufficient scan coverage at the femoral head and a disconnected kidney due to erosion) are both patch-like failures. No evidence is provided that the inaccuracy score responds to systematic auto-segmentation errors such as global boundary contraction or expansion, smooth boundary shifts, or organ confusion. Because the DAE is trained only to undo patch noise, it may treat a smoothly deformed but still plausible mask as a clean shape and reconstruct it with low Dice loss, yielding a low inaccuracy score. This is a load-bearing gap between the training noise distribution and the scope of the central claim; the authors should either add experiments with systematic error types or temper the generality claim.","section":"Section 2.4, Figure 4"},{"comment":"The statistical baseline is implemented with only six features (volume, surface area, surface area to volume ratio, elongation, roundness, and centroid distance), whereas Hui et al. calculate 25 features and Altman et al. include intensity-based metrics. The paper's conclusion that the proposed method achieves 'superior performance compared to existing solutions in the literature' therefore rests on a comparison with a reduced and potentially weaker baseline. This is not evidence against the method itself, but it undermines the literal claim of superiority over prior work. The authors should either implement a fuller feature set or rephrase the comparison claim to state that the DAE outperforms their specific implementation.","section":"Section 2.5, Figure 3"},{"comment":"The CT kidney results (Figure 3) show that the four methods are comparable, with the DAE ranked second for the right kidney behind VAE-Single. The abstract's 'superior performance for the majority of organs' and the conclusion's claim that the method is 'independent of the imaging modality' are therefore stronger than what the evidence supports. The CT kidney use case is the only evaluation beyond MR pelvis, and it uses different organs and segmentation models, so it does not isolate the imaging modality as the varying factor. The authors should either add a same-organ-across-modality experiment or explicitly acknowledge that the evidence for modality independence is indirect.","section":"Section 3.2, Abstract"}],"minor_comments":[{"comment":"There is a typo: 'in dependent' should be 'independent'.","section":"Abstract"},{"comment":"The test set is augmented by rotating each scan by +/-10 degrees, creating multiple test samples from the same patient. This introduces correlation that may affect the bootstrap confidence intervals; the authors should clarify whether the bootstrap resampling accounts for patient identity.","section":"Section 2.3"},{"comment":"The noise-generation hyperparameters are 'optimized manually per organ', but the chosen values are not reported. For reproducibility, the authors should list the hyperparameters in a table or appendix.","section":"Section 2.4"},{"comment":"The organ-specific Dice thresholds are stated without justification or sensitivity analysis. A brief rationale or a reference supporting these thresholds would strengthen the evaluation.","section":"Section 3.1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's core idea—train a denoising autoencoder on clean ground-truth organ masks, then score any auto-segmentation by its Dice loss against the DAE reconstruction—is straightforward and works in the two use cases they present. The multi-organ DAE, the explicit MR and CT demonstration, and the visual reconstructions for explainability are real, if incremental, extensions of Sandfort et al.'s VAE approach. The writing is clear and the comparison against VAE and statistical baselines is a nice touch.\n\nWhere it gets soft is the noise model. The DAE is trained to undo random patch additions/removals on ground-truth masks. That maps naturally to the femoral-head coverage gap and the disconnected kidney shown in their Figure 4, but not obviously to the systematic errors a clinician would care about: global boundary contraction or expansion, smooth shifts, or organ confusion. For those, the DAE may reconstruct a shape close to the input, producing a low inaccuracy score. The paper claims general detection of inaccurate segmentations, but the evidence only supports patch-like errors. That is an internal gap, not just a disagreement with reviewer taste.\n\nThe baseline comparison also slightly oversells the result. Their statistical baseline uses 6 features instead of the 25 in Hui et al., so calling their method 'superior' against a weakened baseline is not a strong statement. On CT kidneys the method is comparable to VAE-Single, not clearly better. No code or data are released, so the numbers are hard to verify independently. The MR test augmentation (rotating scans) is a bit contrived, but they did use 45 original cases, so it is not a fatal flaw.\n\nOverall, this is a reasonable application paper with a plausible mechanism and a clear clinical need. It deserves a serious peer review, but the reviewers should push for validation on a broader set of error modes and a fairer statistical baseline. If the authors can show the DAE catches non-patch errors, the claim of generality would hold up. As it stands, the paper is a useful demonstration for the subfield, not a definitive solution.\n\nI'd bring it to a reading group if the topic is QA in medical segmentation. I wouldn't cite it myself unless continuing that line of work.","headline":"A simple DAE-based QA method for organ segmentations that works in two clinical use cases, but the general detection claim rests on a noise model that only simulates patch-like errors.","tokens_in":7665,"tokens_out":2852,"would_cite":false,"duration_ms":28441,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A denoising autoencoder trained only on correct organ masks can flag inaccurate auto-segmentations across MR and CT by measuring the Dice loss between each mask and its reconstruction.","keywords":["denoising autoencoder","organ at risk segmentation","segmentation quality assurance","radiation therapy planning","explainability","Dice loss","modality independence","medical image segmentation"],"falsifier":"Build a test set of auto-segmentations with deliberately systematic errors (uniform erosion or dilation, global translation, or one organ's label placed where another belongs) at Dice values near the existing thresholds, and check whether the inaccuracy score separates them from accurate masks; if the ranking is no better than random, the random-patch proxy is not faithful.","tokens_in":6716,"feed_emoji":"🩻","tokens_out":8907,"duration_ms":79491,"temperature":0.7,"pith_summary":"The paper claims that a denoising autoencoder trained only on correct, ground-truth organ masks can act as a post-hoc quality check for automatic organ segmentations in radiation therapy planning. At inference, an organ mask is scored by the Dice loss, a standard overlap-based error measure, between the mask and the mask the autoencoder reconstructs; a high score flags a likely inaccurate contour, and the reconstruction itself shows where the contour is wrong. Applied to seven pelvic organs on MR scans and to kidneys on CT scans, the method outperforms variational-autoencoder and statistical baselines on most organs, and it needs neither the segmentation model nor image intensities. The wider claim is that this makes the detector modality-independent and explainable enough for clinical quality assurance.","feed_headline":"Denoising autoencoder flags bad organ contours on MR and CT","feed_subtitle":"Trained only on correct masks, it flags a contour by the gap between it and the rebuilt version.","key_machinery":"The load-bearing mechanism is the denoising autoencoder: a 3D U-Net with residual units trained channel-wise on stacked organ masks, one channel per organ. Training corrupts ground-truth masks by adding or removing random binary patches, with patch hyperparameters tuned per organ so that the signed Dice coefficients of the noisy inputs cover $[-1,1]$ roughly uniformly. At inference, the inaccuracy score is the Dice loss between the preprocessed auto-segmentation and the reconstruction. This carries the entire argument: the difference between what the network expects from correct masks and what it is given is the detection signal, and the reconstruction itself is the explanation.","core_discovery":"The central discovery is that a 3D denoising autoencoder, trained exclusively on ground-truth organ masks corrupted by adding or removing random binary patches, learns a prior over plausible organ geometry, and the Dice loss between an unseen auto-segmentation and its denoised reconstruction is a useful inaccuracy signal. The paper demonstrates this on the MR pelvis use case with seven organs and on the CT kidney use case, showing that the detector is independent of imaging modality and of the segmentation model that produced the masks. In the reported experiments it achieves the highest AUROC and AUPR among the four compared methods for all seven MR pelvis organs and for the left kidney, and ranks second for the right kidney behind a single-organ variational autoencoder. The reconstruction also gives a visual map of where the contour is suspected to be wrong, such as an under-segmented femoral head or a disconnected kidney.","pith_inferences":["If the random-patch training noise is a poor proxy for systematic errors, such as uniform boundary shifts or organ confusion, the DAE may score those errors as plausible; a dedicated test on systematically corrupted segmentations would settle this.","The difference map between an input auto-segmentation and its reconstruction could be used not only to flag errors but to suggest where a clinician should edit the contour, although the paper does not test that editing workflow.","Manual per-organ tuning of the noise hyperparameters would become a bottleneck on extensive multi-organ deployments, so automating that calibration is a natural next step.","The same reconstruction-vs-input principle could transfer to other structured outputs with learnable shape priors, such as vessel trees or bone surfaces, beyond organ masks."],"forward_implications":["A trained detector can be applied post hoc to any segmentation method for the same organs, without retraining or access to the segmentation model's internal features.","Because only binary masks are used, the same detector transfers across imaging modalities, as demonstrated on MR and CT, and could be extended to other modality pairs.","The reconstruction image gives clinicians a direct view of suspicious regions, supporting explainable quality assurance in automated treatment planning workflows.","The method can be positioned as a post-processing step in radiation therapy planning to flag contours that need review, potentially counteracting automation bias."],"supporting_citations":[{"why":"Supplies the single-organ variational autoencoder baseline that the proposed method must beat.","marker":"[6]"},{"why":"Supplies the statistical feature-based quality-assurance approach used as a comparison baseline.","marker":"[2]"},{"why":"Supplies the statistical contour-quality framework with shape and intensity features used as another baseline.","marker":"[5]"},{"why":"Supplies the MR pelvis segmentation models and the manually annotated ground-truth segmentations used for training and evaluation.","marker":"[7]"},{"why":"Supplies the CT kidney auto-segmentations used in the CT use case.","marker":"[9]"},{"why":"Supplies 20 public CT test scans used in the CT kidney evaluation.","marker":"[11]"},{"why":"Supplies the Dice-threshold scheme used to label test auto-segmentations as accurate or inaccurate and a related out-of-distribution detection approach.","marker":"[4]"},{"why":"Supplies the residual U-Net architecture used to build the denoising autoencoder.","marker":"[12]"}],"fun_headline_variants":["Denoiser that learned good anatomy flags faulty organ contours","Reconstruction error spots organ segmentation mistakes on MR/CT","Modality-independent autoencoder flags bad organ masks via rebuild","Train on good masks only, let reconstruction reveal errors","Denoiser's rebuild gap uncovers wrong organ contours on CT and MR"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The training corruption, which adds or removes random binary patches from ground-truth masks, is assumed to resemble the errors real auto-segmentation systems make; if real errors are systematic boundary shifts or organ confusions rather than patch-like defects, the learned inaccuracy score may not rise for them.","fun_headline_variants_meta":{"raw":{"variants":["Denoiser that learned good anatomy flags faulty organ contours","Reconstruction error spots organ segmentation mistakes on MR/CT","Modality-independent autoencoder flags bad organ masks via rebuild","Train on good masks only, let reconstruction reveal errors","Denoiser's rebuild gap uncovers wrong organ contours on CT and MR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000899,"raw_usage":{"total_tokens":3825,"prompt_tokens":854,"completion_tokens":2971,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":470,"completion_tokens_details":{"reasoning_tokens":2887}},"tokens_in":470,"tokens_out":2971,"duration_ms":18895,"temperature":1.0,"reasoning_tokens":2887,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:34:58.111586+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a test set of auto-segmentations with deliberately systematic errors (uniform erosion or dilation, global translation, or one organ's label placed where another belongs) at Dice values near the existing thresholds, and check whether the inaccuracy score separates them from accurate masks; if the ranking is no better than random, the random-patch proxy is not faithful.","supporting_citations":[{"cited_title":"We thank the annotation team, including authors Zs.K., B.D","cited_arxiv_id":null,"evidence_quote":"Supplies the single-organ variational autoencoder baseline that the proposed method must beat."},{"cited_title":"MR Pelvis Segmentation Models To generate organ auto-segmentations for the MR pelvis use case, we utilized deep learning segmentation models described in Czipczer et al","cited_arxiv_id":null,"evidence_quote":"Supplies the statistical feature-based quality-assurance approach used as a comparison baseline."},{"cited_title":"Ethical statements can be found in Section 2.8 in Czipczer et al","cited_arxiv_id":null,"evidence_quote":"Supplies the statistical contour-quality framework with shape and intensity features used as another baseline."},{"cited_title":"We demonstrated that our method provides superior performance compared to existing solutions in the literature","cited_arxiv_id":null,"evidence_quote":"Supplies the Dice-threshold scheme used to label test auto-segmentations as accurate or inaccurate and a related out-of-distribution detection approach."},{"cited_title":"Use of Variational Autoencoders with Unsupervised Learning to Detect Incorrect Organ Segmentations at CT,","cited_arxiv_id":null,"evidence_quote":"Supplies the residual U-Net architecture used to build the denoising autoencoder."}],"review_version":1}