REVIEW 4 major objections 6 minor 20 references
Synthetic Poisoning Attacks: The Impact of Poisoned MRI Image on U-Net Brain Tumor Segmentation
T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read Adding GAN-generated MRI to a U-Net training set lowers brain-tumor segmentation quality, with Dice falling from 0.8937 at 33% synthetic data to 0.7474 at 83%.
desk verdict A useful question undermined by a confounded protocol and unsupported claims. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the poisoned training set $D'(p) = D \cup X'$, where $X'$ are synthetic T1-contrast-enhanced MRI produced by a GAN-based cross-domain CT-to-MRI translation model with a shared encoding-decoding structure and shortest-path regularization. Varying the synthetic proportion $p \in \{16.67\%, 33.33\%, 50\%, 66.67\%, 83.33\%\}$ while keeping the real validation set fixed isolates, in the authors' design, the amount of contamination the U-Net is exposed to. The mechanism is downstream: fake images enter training, and the segmentation metrics on real test images—Dice, Jaccard, accuracy, sensitivity—reveal how much the model's tumor boundary predictions drift.
What would settle it
Train two U-Nets at 83.33% synthetic content with identical total size, one on real MRIs and one on synthetic MRIs paired with the same ground-truth masks; if Dice on the real validation set does not fall to roughly 0.7474 in the synthetic-conditioned model, the effect is not specifically synthetic image content. A cheaper check is to inspect the synthetic masks: if the fake MRIs inherit masks from the source CTs, retraining with manually corrected labels would test whether misalignment, not realism, causes the drop.
Extended reading notes
Core claim
The paper's central discovery is a quantitative degradation curve: as the proportion of synthetic MRI in the training mixture rises, a U-Net's real-image segmentation quality falls. Formally, the authors hypothesize $\mathrm{Dice}(M') < \mathrm{Dice}(M)$, where $M$ is trained on real data only and $M'$ on $D \cup X'$. Their experiments support this most clearly at high contamination levels: the worst condition, $p = 83.33\%$, gives Dice 0.7474 and sensitivity 0.7577, against 0.8939 and 0.9136 for real-only training. Accuracy remains near 0.998 throughout, so the damage is concentrated in overlap and recall, i.e., in the model's ability to delineate tumor boundaries rather than to classify background pixels. The authors conclude that synthetic content should be regulated and that controlled low-proportion augmentation may be acceptable.
Load-bearing premise
The load-bearing premise is that the observed performance drop is caused by the synthetic proportion of the training set; the poisoned sets are larger than the baseline $D$, and the paper never states how segmentation masks are assigned to the synthetic MRIs, so label misalignment or dataset size, rather than synthetic content itself, could drive the degradation.
Editorial extensions
If this is right
- Above 50% synthetic content, segmentation quality drops steeply: at 83.33% the mean Dice falls to 0.7474 and sensitivity to 0.7577, meaning the model misses substantially more tumor tissue.
- Up to about one-third synthetic content, mean Dice (0.8937 at 33.33%) stays close to the real-only baseline (0.8939), suggesting a tolerable range for augmentation.
- Pixel accuracy remains near 0.998 across conditions, so auditing synthetic augmentation with accuracy alone would miss the damage; overlap and recall metrics are needed.
- Medical pipelines should screen synthetic samples and cap their fraction, because unregulated generative augmentation can behave like a silent poisoning attack.
Reading between the lines
- The paper's poisoning account would be sharper if it ruled out label-source effects; a natural follow-up is to compare synthetic MRIs labeled by the generator's source CT against the same synthetic images labeled by a human expert, holding image content fixed.
- Because the 16.67% condition shows a mean Dice (0.8650) below the 33.33% condition (0.8937) with notably larger standard deviation, the monotonic-trend reading rests mainly on the later proportions; a safe-threshold claim would need more replicates.
- The result suggests that quality control should target semantic fidelity—whether the anatomy in the generated image matches its label—rather than visual realism alone; this is testable by having radiologists rate realism and correlating those ratings with Dice drops.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript investigates whether adding GAN-generated synthetic T1-contrast-enhanced (T1-Ce) MRI scans to the training set degrades U-Net brain tumor segmentation. The authors use a cross-domain image translation model from Xie et al. (2023) to generate 150 synthetic T1-Ce images, train a baseline U-Net on the real BraTS2021 dataset, and train 'poisoned' U-Nets on mixtures with synthetic proportions p = 16.67% to 83.33%. Performance is measured with Dice, Jaccard, accuracy, and sensitivity on a real MRI test set. The paper concludes that increasing synthetic data proportion causes significant segmentation degradation, citing a Dice drop from 0.8937 at p=33.33% to 0.7474 at p=83.33%, and states that accuracy and sensitivity show similar downward trends.
Significance. The paper addresses a timely and practically relevant question: whether synthetic data augmentation in medical imaging can act as a form of data poisoning. Its use of the public BraTS2021 benchmark and standard segmentation metrics is appropriate, and the metric equations are clearly stated. The appendix provides qualitative examples of synthetic MRIs. However, the study's internal validity is severely compromised: the poisoned-dataset construction confounds synthetic proportion with total training-set size, and the labeling procedure for synthetic images is never specified. The reported numbers in Table 1 also do not support the claimed monotonic degradation, making the central causal conclusion unsupported as presented.
major comments (4)
- [Section 4.1, Algorithm 1] The construction D′(p) ← D ∪ X′ confounds the synthetic proportion p with the total training-set size, and the paper never specifies how p is realized. For a fixed D of 150 real scans and X′ of 150 synthetic scans, p=50% corresponds to D ∪ X′ (300 images), whereas p=16.67% and p=83.33% require either subsampling D or X′ or a different mixing procedure that is not described. Any performance difference across p could therefore be caused by the number of training examples rather than by the proportion of synthetic content. In addition, the paper does not state how the enhanced tumor (ET) masks are assigned to the 150 synthetic T1-Ce images. If the masks are inherited from the source CT or a paired real MRI, GAN-induced geometric or contrast inconsistencies become label noise that scales with p, providing an alternative explanation for the observed Dice drop. The central causal claim requires controlling both variables, and the current protocol does not do so.
- [Table 1] The data in Table 1 do not support the abstract's claim of 'significant performance degradation as synthetic data increases.' Dice is 0.8939 at p=0, falls to 0.8650 at p=16.67%, then rises to 0.8937 at p=33.33% before declining to 0.7474 at p=83.33%; the 16.67% condition is lower than the 33.33% condition, contrary to a monotonic dose-response relationship. Sensitivity increases from 0.9136 (p=0) to 0.9292 (p=50%), and accuracy is almost constant (0.9983, 0.9981, 0.9979, 0.9978, 0.9967) apart from an anomalous 0.9638 at p=16.67%. Consequently, the statement that 'accuracy and sensitivity exhibit similar downward trends' is contradicted by the table.
- [Abstract and Section 4.2] The abstract's headline comparison selects the starting point after the initial non-monotonic variation: the drop is reported from 0.8937 (33.33% synthetic) to 0.7474 (83.33% synthetic), which excludes the p=16.67% row. If the trend were presented from the baseline p=0, the pattern would be non-monotonic (0.8939, 0.8650, 0.8937, 0.8572, 0.8146, 0.7474). The claim that models with p ≤ 33.33% 'maintain relatively stable performance' is also inconsistent with the 0.8650 Dice at p=16.67%, which is already below the baseline. The paper should present the full data without selective endpoints or revise the conclusion accordingly.
- [Table 1 and Section 4.2] No statistical support is provided for the word 'significant.' The table reports means and standard deviations but no number of independent training runs, p-values, confidence intervals, or effect sizes. The standard deviations are large and overlapping across conditions (e.g., Dice at p=0 is 0.8939±0.1243 vs. 0.8572±0.1580 at p=50%), so the observed differences may not be statistically reliable. Without repeated runs or a significance test, the central conclusion of significant degradation is not established.
minor comments (6)
- [Figure 3] Figure 3 includes a 'Poisoning Rate: 100%' panel, but Table 1 reports results only up to p=83.33%; the figure should be reconciled with the table or the missing condition should be reported.
- [Introduction] The Introduction describes the GAN as 'a novel GAN-based model (Xie et al., 2023)'; since the GAN is taken verbatim from Xie et al. (2023), it should not be called novel in this work. Please rephrase to avoid implying novelty.
- [Section 4.1] The paper does not describe the U-Net training details (number of epochs, optimizer, learning rate, loss function, data preprocessing, train/validation/test split), which are needed for reproducibility and for assessing whether the observed variability might be due to training randomness.
- [Section 2] In the Data Poisoning Attack paragraph, the sentence 'poisoning can occur through mislabeled (Tolpegin et al., 2020; Lin et al., 2021), perturbed (Martinelli et al., 2023; Bortsova et al., 2021)' is grammatically incomplete and should be rewritten.
- [Section 4.1] The data lineage is unclear: the GAN is trained on 660 paired CT-MRI datasets, but the relationship of these pairs to the 150 BraTS2021 patients whose T1-Ce scans are used for training is not explained. Please clarify whether the synthetic MRIs are generated from the same patients or from a separate cohort.
- [Appendix A] The appendix shows qualitative examples of fake MRI but provides no quantitative assessment of synthetic image quality (e.g., FID, SSIM, or a reader study), which would help interpret how label misalignment might arise.
Circularity Check
Empirical measurement study with no derivation chain; no circularity identified.
full rationale
The paper is an empirical measurement study rather than a derivation from first principles. Its central claim, Dice(M') < Dice(M), is stated as a testable expectation and is then evaluated experimentally, not derived from a fitted parameter or from the definition of the synthetic proportion. The metrics used (Dice, Jaccard, accuracy, sensitivity) are standard definitions and do not encode the conclusion. The generative model is attributed to Xie et al. (2023), whose authors are not the present authors, so no load-bearing self-citation chain is used to force the result. The cited own-papers (e.g., Li et al. 2024a, Zheng et al. 2024) appear only in a general reference list and do not support the poisoning claim. The experimental protocol does change more than the synthetic proportion: D'(p) = D ∪ X' also changes training-set size, and the paper does not specify how segmentation masks are assigned to the generated synthetic MRIs. These are validity concerns about causal attribution, not circularity: they do not make any prediction equivalent to an input by construction. No parameter is fitted to the outcome and then renamed as a prediction, and no result is imported from the authors' own prior work as an external theorem. Accordingly, the circularity burden is minimal and the score is 0.
Assumptions & free parameters
assumptions (3)
- domain assumption The GAN-generated synthetic MRI images are labeled with the same segmentation masks as their source real images.
- domain assumption The BraTS2021 T1-Ce subset and the trained U-Net reflect typical clinical tumor segmentation conditions.
- domain assumption U-Net performance is stable across random initializations; reported means are representative.
Cite this review
Pith. "Pith review of Synthetic Poisoning Attacks: The Impact of Poisoned MRI Image on U-Net Brain Tumor Segmentation." pith.science (2026). https://pith.science/paper/MHSD425D
@misc{pith2026250203825,
author = {Pith},
title = {Pith review of: Synthetic Poisoning Attacks: The Impact of Poisoned MRI Image on U-Net Brain Tumor Segmentation},
year = {2026},
howpublished = {\url{https://pith.science/paper/MHSD425D}},
note = {Machine review of arXiv:2502.03825}
}
read the original abstract
Deep learning-based medical image segmentation models, such as U-Net, rely on high-quality annotated datasets to achieve accurate predictions. However, the increasing use of generative models for synthetic data augmentation introduces potential risks, particularly in the absence of rigorous quality control. In this paper, we investigate the impact of synthetic MRI data on the robustness and segmentation accuracy of U-Net models for brain tumor segmentation. Specifically, we generate synthetic T1-contrast-enhanced (T1-Ce) MRI scans using a GAN-based model with a shared encoding-decoding framework and shortest-path regularization. To quantify the effect of synthetic data contamination, we train U-Net models on progressively "poisoned" datasets, where synthetic data proportions range from 16.67% to 83.33%. Experimental results on a real MRI validation set reveal a significant performance degradation as synthetic data increases, with Dice coefficients dropping from 0.8937 (33.33% synthetic) to 0.7474 (83.33% synthetic). Accuracy and sensitivity exhibit similar downward trends, demonstrating the detrimental effect of synthetic data on segmentation robustness. These findings underscore the importance of quality control in synthetic data integration and highlight the risks of unregulated synthetic augmentation in medical image analysis. Our study provides critical insights for the development of more reliable and trustworthy AI-driven medical imaging systems.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[5]
Gan-based synthetic brain mr image generation
Changhee Han, Hideaki Hayashi, Leonardo Rundo, Ryosuke Araki, Wataru Shimoda, Shinichi Mu- ramatsu, Yujiro Furukawa, Giancarlo Mauri, and Hideki Nakayama. Gan-based synthetic brain mr image generation. In 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), pp. 734–738. IEEE,
work page 2018
-
[7]
Synthetic data in ai: Challenges, applications, and ethical implications
Shuang Hao, Wenfeng Han, Tao Jiang, Yiping Li, Haonan Wu, Chunlin Zhong, Zhangjun Zhou, and He Tang. Synthetic data in ai: Challenges, applications, and ethical implications. arXiv preprint arXiv:2401.01629,
-
[8]
Safety and trustworthiness of deep neural networks: A survey
Xiaowei Huang, Daniel Kroening, Marta Kwiatkowska, Wenjie Ruan, Youcheng Sun, Emese Thamo, Min Wu, and Xinping Yi. Safety and trustworthiness of deep neural networks: A survey. arXiv preprint arXiv:1812.08342, pp. 151,
-
[9]
Ming Kang, Fung Fung Ting, Rapha ¨el C-W Phan, Zongyuan Ge, and Chee-Ming Ting. A mul- timodal feature distillation with cnn-transformer network for brain tumor segmentation with in- complete modalities. arXiv preprint arXiv:2404.14019,
-
[10]
Auto-encoding variational bayes
Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114,
-
[11]
Tianhao Li, Jingyu Lu, Chuangxin Chu, Tianyu Zeng, Yujia Zheng, Mei Li, Haotian Huang, Bin Wu, Zuoxian Liu, Kai Ma, et al. Scisafeeval: a comprehensive benchmark for safety alignment of large language models in scientific tasks. arXiv preprint arXiv:2410.03769, 2024a. Tianhao Li, Yujia Zheng, Weizhi Ma, Guangshuo Wang, Zhengping Li, and Lijun Wang. P-4.33...
-
[13]
Data poi- soning attacks over diabetic retinopathy images classification
Fabio Martinelli, Francesco Mercaldo, Marcello Di Giammarco, and Antonella Santone. Data poi- soning attacks over diabetic retinopathy images classification. In 2023 IEEE International Con- ference on Big Data (BigData), pp. 3698–3703. IEEE,
work page 2023
-
[14]
The multimodal brain tumor image segmentation benchmark (brats)
Bjoern H Menze, Andras Jakab, Stefan Bauer, Jayashree Kalpathy-Cramer, Keyvan Farahani, Justin Kirby, Yuliya Burren, Nicole Porz, Johannes Slotboom, Roland Wiest, et al. The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging , 34 (10):1993–2024,
work page 1993
Show all 20 references
-
[15]
Conditional generative adversarial nets
Mehdi Mirza. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784,
-
[17]
U-net: Convolutional networks for biomed- ical image segmentation
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medical image computing and computer-assisted intervention– MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceed- ings, ...
2015
-
[18]
A survey on trustworthiness in foundation models for medical image analysis
Congzhen Shi, Ryan Rezai, Jiaxi Yang, Qi Dou, and Xiaoxiao Li. A survey on trustworthiness in foundation models for medical image analysis. arXiv preprint arXiv:2407.15851,
-
[19]
Intriguing properties of neural networks
C Szegedy. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199,
-
[2014]
Medical image synthesis with context-aware generative adversarial networks
Dong Nie, Roger Trullo, Jun Lian, Caroline Petitjean, Su Ruan, Qian Wang, and Dinggang Shen. Medical image synthesis with context-aware generative adversarial networks. In Medical Im- age Computing and Computer Assisted Intervention- MICCAI 2017: 20th International Confer- enc...
2017
-
[2018]
Robust machine learning systems: Reliability and security for deep neural networks
Muhammad Abdullah Hanif, Faiq Khalid, Rachmad Vidya Wicaksana Putra, Semeen Rehman, and Muhammad Shafique. Robust machine learning systems: Reliability and security for deep neural networks. In 2018 IEEE 24th international symposium on on-line testing and robust system design ...
2018
-
[2020]
A survey on data poisoning attacks and defenses
Jiaxin Fan, Qi Yan, Mohan Li, Guanqun Qu, and Yang Xiao. A survey on data poisoning attacks and defenses. In 2022 7th IEEE International Conference on Data Science in Cyberspace (DSC), pp. 48–55. IEEE,
2022
-
[2021]
Active learning under malicious mislabeling and poisoning attacks
Jing Lin, Ryan Luley, and Kaiqi Xiong. Active learning under malicious mislabeling and poisoning attacks. In 2021 IEEE global communications conference (GLOBECOM), pp. 1–6. IEEE,
2021
-
[2022]
Trustworthy deep learning for medical image segmentation
7 Preprint Lucas Fidon. Trustworthy deep learning for medical image segmentation. arXiv preprint arXiv:2305.17456,
-
[2023]
Poi- sonbench: Assessing large language model vulnerability to data poisoning
Tingchen Fu, Mrinank Sharma, Philip Torr, Shay B Cohen, David Krueger, and Fazl Barez. Poi- sonbench: Assessing large language model vulnerability to data poisoning. arXiv preprint arXiv:2410.08811,
-
[2024]
Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge
Spyridon Bakas, Mauricio Reyes, Andras Jakab, Stefan Bauer, Markus Rempfler, Alessandro Crimi, Russell Takeshi Shinohara, Christoph Berger, Sung Min Ha, Martin Rozycki, et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment...
-
[2025]
Data poisoning attacks against federated learning systems
Vale Tolpegin, Stacey Truex, Mehmet Emre Gursoy, and Ling Liu. Data poisoning attacks against federated learning systems. In Computer security–ESORICs 2020: 25th European symposium on research in computer security, ESORICs 2020, guildford, UK, September 14–18, 2020, proceed- i...
2020
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.