REVIEW 4 major objections 6 minor 1 cited by
Ultrasound Image Generation using Latent Diffusion Models
T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read This paper claims that fine-tuning Stable Diffusion on 780 breast ultrasound images produces synthetic images that look realistic to experts and improve an external classifier from 81% to 87% AUC.
desk verdict Plausible feasibility demo of Stable Diffusion fine-tuning for breast ultrasound, but the one quantitative claim (81% to 87% AUC) is under-specified and should not be treated as evidence of utility yet. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is Stable Diffusion, a latent diffusion model: a variational autoencoder compresses 512x512 images to 64x64 latents, a U-Net learns to denoise these latents, and CLIP text embeddings guide the denoising. The paper fine-tunes this pre-trained model on BUSI, a small dataset of 780 images, which is feasible in a few hours on a single A100 GPU. For spatial control, the paper adds ControlNet, a trainable copy of the network whose outputs are added to the frozen decoder through zero-convolution layers, trained with the segmentation masks from BUSI as the conditioning input. This combination lets the pre-trained generative prior be redirected toward ultrasound while retaining the ability to follow a text prompt and, with ControlNet, a segmentation mask.
What would settle it
Re-running the augmentation experiment with a fully documented protocol, fixed number and selection of generated images, the ResNet-50 recipe from [24], and five random seeds, then reporting per-seed AUC with confidence intervals for the difference between augmented and non-augmented classifiers on B25 would settle the claim; if the difference is not significant, the utility claim as currently stated would be refuted.
Extended reading notes
Core claim
On its own terms, the paper claims that Stable Diffusion v1.5, fine-tuned on the BUSI dataset, can generate high-quality breast ultrasound images from simple text prompts such as "Ultrasound image of a normal/benign/malignant breast", with textures (skin, fat, glandular tissue, muscle, ribs) close to real ultrasound and with benign lesions appearing as round black cysts while malignant lesions are irregular. A quantitative evaluation with a ResNet-50 trained as in [24] found that augmenting the training data with some generated images raised AUC from 81% to 87% when tested on B25, a different breast imaging dataset; the authors interpret this as evidence that the generated samples are useful for downstream classification. Conditioning the fine-tuned generator through ControlNet with segmentation masks produced multiple plausible images that match the input mask, giving the user control over the lesion shape and location.
Load-bearing premise
The entire utility claim rests on the assumption that the reported 81% to 87% AUC improvement is real and not an artifact of an incomplete or biased evaluation; if that protocol was flawed, the central claim that generated images improve classification collapses.
Editorial extensions
If this is right
- A foundation text-to-image model can be steered into a medical imaging domain with a few hundred images and modest compute, without physics-based ultrasound simulation.
- Synthetic breast ultrasound images can be used to enlarge training sets, and the reported external-dataset AUC gain (81% to 87%) suggests they can improve generalization rather than merely memorizing the training distribution.
- Segmentation-mask conditioning makes it possible to generate many plausible ultrasound images for a given lesion shape, which could support studies of shape-dependent classifier behavior and segmentation model training.
- Releasing the code gives other groups a reproducible baseline for ultrasound generation with latent diffusion models, lowering the barrier to applying the same recipe to other organs or pathologies.
Reading between the lines
- A direct extension the paper does not run is a diversity check: computing pairwise similarity or a perceptual distance among generated images, and between generated and BUSI images, would show whether the classifier gain comes from genuine variation or from near-duplicates of common BUSI patterns.
- Repeating the 81% to 87% AUC comparison with a fully specified protocol and confidence intervals would convert the reported gain from a single number into an established effect size.
- Because the conditioning is mask-based, the same pipeline could synthesize lesion shapes drawn from another distribution, such as physics-based simulations or clinical sketches, blending generative realism with controlled anatomy; the paper mentions this direction only as future work.
- The same fine-tuning recipe likely transfers to other ultrasound domains with small public datasets, such as cardiac or lung ultrasound, but this transfer is not tested in the paper.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes fine-tuning Stable Diffusion v1.5 on the BUSI breast ultrasound dataset to generate realistic ultrasound images from text prompts, and extends this with ControlNet to condition generation on segmentation masks. The authors report that the generated images appeared realistic to three experienced ultrasound scientists and a radiologist, and they provide two quantitative results: (1) augmenting a training set with some generated images improved ResNet-50 AUC on a separate B25 dataset from 81% to 87%, and (2) a classifier trained on images generated by a model fine-tuned on 20% of BUSI achieved 94% AUC on the generator's training set. The paper argues this demonstrates the feasibility of adapting large latent diffusion models to small medical imaging datasets with modest computational cost.
Significance. If the quantitative claims are substantiated, this work would offer a practical and timely contribution: a pipeline for generating synthetic ultrasound images to augment small datasets, with user control through segmentation masks. The approach of fine-tuning a large pre-trained latent diffusion model rather than training from scratch is sensible, and the reported single-GPU fine-tuning cost is attractive. The qualitative examples are plausible and suggest the method captures key ultrasound texture and lesion morphology. However, the central utility claim currently rests on an under-specified evaluation and a self-referential accuracy figure, so the significance is conditional on additional evidence.
major comments (4)
- [Section 3.1.2] The 81% to 87% AUC improvement on B25 is the central quantitative claim, but the evaluation protocol is under-specified. The paper does not report the size and class balance of B25, the preprocessing applied, the number of generated images added, the criteria behind 'some of the generated images,' the number of training seeds, or confidence intervals/statistical tests. Without these details, the reader cannot assess whether the 6-point AUC gain is statistically meaningful or reproducible. Please provide a complete experimental protocol or explicitly label this as a preliminary observation.
- [Section 3.1.2] The improvement from 81% to 87% AUC is not attributed to the synthetic images specifically because no matched-size control is reported. Adding an equivalent number of real BUSI images (or even repeated real images) to the training set might produce a similar gain. Please include a control experiment with real-image augmentation of the same size, and report per-seed variance to distinguish the effect of synthetic images from the effect of simply increasing training data size.
- [Section 3.1.2] The 94% AUC result is self-referential: the classifier is tested on the exact subset of BUSI used to fine-tune the generator. High accuracy on this set mainly demonstrates that the generator has memorized or closely reproduced its training distribution, not that the generated images generalize to unseen data. Please remove this result or re-frame it as an internal consistency check, and evaluate on a held-out subset or an external dataset if generalization is intended.
- [Section 3.1.1] The realism claim rests on informal expert opinion, but no evaluation methodology is described. Please report how many images were assessed, whether the experts were blinded to the source (generated vs. real), what instructions they received, and whether there was any inter-rater agreement metric. A forced-choice or Turing-style test against real BUSI images would substantially strengthen this claim.
minor comments (6)
- [Section 2.1] The term 'U-NET' should be 'U-Net' for consistency with the literature, and 'VAE' is used without expanding the abbreviation at first use.
- [Section 3.1.2] The B25 dataset is not defined; please add a citation and describe how it was obtained, its size, and its preprocessing.
- [Section 3.1.2] The number of generated images used for augmentation is not stated; please specify the total and per-class counts, as well as the selection procedure.
- [Section 3.1.3] The sentence ending with 'The Figure' at the end of Section 3.1.3 appears incomplete; please proofread and complete the reference to Figure 4.
- [Section 2.1] The paper mentions fine-tuning 'in a few hours using a single A100 GPU' but does not report exact training time, batch size, learning rate, or number of steps; adding these would aid reproducibility.
- [Abstract] The code release URL (http://code.sonography.ai/) is mentioned but no repository link or license is provided; please include an active and permanent link.
Circularity Check
No significant circularity; the B25 benchmark is independent and the 94% self-check is not used as external evidence.
full rationale
The central quantitative claim—augmenting training with Stable Diffusion-generated breast ultrasound images improves ResNet-50 AUC on B25 from 81% to 87%—rests on an external dataset (B25) that is not used to fine-tune or fit the generator, so the evaluation is not circular by construction. The 94% AUC result is explicitly described as predicting the generative model's own training set; it is a self-referential sanity check and is not used as evidence of external generalization, so it does not form part of a circular derivation chain. The only self-citation, reference [24], is used to specify the ResNet-50 training setup, not to justify the validity of the generated images or the B25 improvement, and is therefore not load-bearing. The under-specified B25 protocol (missing test-set size, preprocessing, number of generated images, and error bars) is a correctness and reproducibility limitation, not a circularity. No equation or parameter in the paper is defined in terms of the quantity it supposedly predicts.
Assumptions & free parameters
free parameters (1)
- BUSI training subset for the single-classifier experiment =
20% of BUSI
assumptions (4)
- domain assumption Stable Diffusion v1.5 and ControlNet pre-trained weights are appropriate foundation models for ultrasound image generation
- domain assumption BUSI dataset is representative of breast ultrasound images
- domain assumption Visual assessment by three experienced US scientists and a US radiologist is a reliable measure of realism
- domain assumption The B25 dataset and the ResNet-50 training setup from reference [24] are correctly applied in the external evaluation
Cite this review
Pith. "Pith review of Ultrasound Image Generation using Latent Diffusion Models." pith.science (2026). https://pith.science/paper/KKSNBMMI
@misc{pith2026250208580,
author = {Pith},
title = {Pith review of: Ultrasound Image Generation using Latent Diffusion Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/KKSNBMMI}},
note = {Machine review of arXiv:2502.08580}
}
read the original abstract
Diffusion models for image generation have been a subject of increasing interest due to their ability to generate diverse, high-quality images. Image generation has immense potential in medical imaging because open-source medical images are difficult to obtain compared to natural images, especially for rare conditions. The generated images can be used later to train classification and segmentation models. In this paper, we propose simulating realistic ultrasound (US) images by successive fine-tuning of large diffusion models on different publicly available databases. To do so, we fine-tuned Stable Diffusion, a state-of-the-art latent diffusion model, on BUSI (Breast US Images) an ultrasound breast image dataset. We successfully generated high-quality US images of the breast using simple prompts that specify the organ and pathology, which appeared realistic to three experienced US scientists and a US radiologist. Additionally, we provided user control by conditioning the model with segmentations through ControlNet. We will release the source code at http://code.sonography.ai/ to allow fast US image generation to the scientific community.
Figures
Forward citations
Cited by 1 Pith paper
-
Breast Ultrasound Tumor Generation via Mask Generator and Text-Guided Network:A Clinically Controllable Framework with Downstream Evaluation
A text-and-mask guided diffusion framework generates realistic synthetic breast ultrasound tumors, and augmenting real training data with these images improves downstream classification and segmentation across six pub...
Reference graph
Works this paper leans on
-
[1]
High-resolution image synthesis with latent diffusion models,
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B., “High-resolution image synthesis with latent diffusion models,” in [ Proceedings of the IEEE/CVF conference on computer vision and pattern recognition], 10684–10695 (2022)
work page 2022
-
[2]
Dataset of breast ultrasound images,
Al-Dhabyani, W., Gomaa, M., Khaled, H., and Fahmy, A., “Dataset of breast ultrasound images,” Data in brief 28, 104863 (2020)
work page 2020
-
[3]
Adding conditional control to text-to-image diffusion models,
Zhang, L., Rao, A., and Agrawala, M., “Adding conditional control to text-to-image diffusion models,” in [Proceedings of the IEEE/CVF International Conference on Computer Vision], 3836–3847 (2023)
work page 2023
-
[4]
Synthetically enhanced: unveiling synthetic data’s potential in medical imaging research,
Khosravi, B., Li, F., Dapamede, T., Rouzrokh, P., Gamble, C. U., Trivedi, H. M., Wyles, C. C., Sellergren, A. B., Purkayastha, S., Erickson, B. J., et al., “Synthetically enhanced: unveiling synthetic data’s potential in medical imaging research,” EBioMedicine 104 (2024)
work page 2024
-
[5]
Augmenting medical image classifiers with synthetic data from latent diffusion models,
Sagers, L. W., Diao, J. A., Melas-Kyriazi, L., Groh, M., Rajpurkar, P., Adamson, A. S., Rotemberg, V., Daneshjou, R., and Manrai, A. K., “Augmenting medical image classifiers with synthetic data from latent diffusion models,” arXiv preprint arXiv:2308.12453(2023)
arXiv 2023
-
[6]
Make the most of must, an open-source matlab ultrasound toolbox,
Garcia, D., “Make the most of must, an open-source matlab ultrasound toolbox,” in [ 2021 IEEE interna- tional ultrasonics symposium (IUS)], 1–4, IEEE (2021)
work page 2021
-
[7]
Field: A program for simulating ultrasound systems,
Jensen, J. A., “Field: A program for simulating ultrasound systems,” Medical & Biological Engineering & Computing 34(sup. 1), 351–353 (1997)
work page 1997
-
[8]
k-wave: Matlab toolbox for the simulation and reconstruction of photoacoustic wave fields,
Treeby, B. E. and Cox, B. T., “k-wave: Matlab toolbox for the simulation and reconstruction of photoacoustic wave fields,” Journal of biomedical optics15(2), 021314–021314 (2010)
work page 2010
Show all 30 references
-
[9]
Ultraray: Full-path ray tracing for enhancing realism in ultrasound simulation,
Duelmer, F., Azampour, M. F., and Navab, N., “Ultraray: Full-path ray tracing for enhancing realism in ultrasound simulation,” arXiv preprint arXiv:2501.05828(2025)
2025 arXiv
-
[10]
Ultrasound segmentation using u-net: learning from simulated data and testing on real data,
Behboodi, B. and Rivaz, H., “Ultrasound segmentation using u-net: learning from simulated data and testing on real data,” in [ 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC)], 6628–6631, IEEE (2019)
2019
-
[11]
Diffusion as sound propagation: Physics- inspired model for ultrasound image generation,
Dom ´ ınguez, M., Velikova, Y., Navab, N., and Azampour, M. F., “Diffusion as sound propagation: Physics- inspired model for ultrasound image generation,” in [International Conference on Medical Image Computing and Computer-Assisted Intervention], 613–623, Springer (2024)
2024
-
[12]
Echo from noise: synthetic ultra- sound image generation using diffusion models for real image segmentation,
Stojanovski, D., Hermida, U., Lamata, P., Beqiri, A., and Gomez, A., “Echo from noise: synthetic ultra- sound image generation using diffusion models for real image segmentation,” in [ International Workshop on Advances in Simplifying Medical Ultrasound], 34–43, Springer (2023)
2023
-
[13]
Diffusion model-based data augmentation for lung ultrasound classification with limited data,
Zhang, X., Gangopadhyay, A., Chang, H.-M., and Soni, R., “Diffusion model-based data augmentation for lung ultrasound classification with limited data,” in [ ML4H@NeurIPS ], (2023)
2023
-
[14]
Ultrasound image synthesis using generative ai for lung ultrasound detection,
Chou, Y.-C., Li, G. Y., Chen, L., Zahiri, M., Balaraju, N., Patil, S., Hicks, B., Schnittke, N., Kessler, D. O., Shupp, J., et al., “Ultrasound image synthesis using generative ai for lung ultrasound detection,” arXiv preprint arXiv:2501.06356 (2025)
2025 arXiv
-
[15]
Echonet-synthetic: Privacy-preserving video generation for safe medical data sharing,
Reynaud, H., Meng, Q., Dombrowski, M., Ghosh, A., Day, T., Gomez, A., Leeson, P., and Kainz, B., “Echonet-synthetic: Privacy-preserving video generation for safe medical data sharing,” in [ International Conference on Medical Image Computing and Computer-Assisted Intervention]...
2024
-
[16]
Ultrasound image enhancement with the variance of diffusion models,
Zhang, Y., Huneau, C., Idier, J., and Mateus, D., “Ultrasound image enhancement with the variance of diffusion models,” in [ 2024 IEEE Ultrasonics, Ferroelectrics, and Frequency Control Joint Symposium (UFFC-JS)], 1–5, IEEE (2024)
2024
-
[17]
Denoising plane wave ultrasound images using diffusion probabilistic models,
Asgariandehkordi, H., Goudarzi, S., Sharifzadeh, M., Basarab, A., and Rivaz, H., “Denoising plane wave ultrasound images using diffusion probabilistic models,” IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control(2024)
2024
-
[18]
Dehazing ultrasound using diffusion models,
Stevens, T. S., Meral, F. C., Yu, J., Apostolakis, I. Z., Robert, J.-L., and Van Sloun, R. J., “Dehazing ultrasound using diffusion models,” IEEE Transactions on Medical Imaging(2024)
2024
-
[19]
Deep ultrasound denoising using diffusion probabilistic models,
Asgariandehkordi, H., Goudarzi, S., Basarab, A., and Rivaz, H., “Deep ultrasound denoising using diffusion probabilistic models,” in [ 2023 IEEE International Ultrasonics Symposium (IUS)], 1–4, IEEE (2023)
2023
-
[20]
Denoising diffusion probabilistic models,
Ho, J., Jain, A., and Abbeel, P., “Denoising diffusion probabilistic models,” Advances in neural information processing systems33, 6840–6851 (2020)
2020
-
[21]
Diffusion models beat gans on image synthesis,
Dhariwal, P. and Nichol, A., “Diffusion models beat gans on image synthesis,” Advances in neural informa- tion processing systems34, 8780–8794 (2021)
2021
-
[22]
Learning transferable visual models from natural language supervision,
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al., “Learning transferable visual models from natural language supervision,” in [ International conference on machine learning], 8748–8763, PMLR (2021)
2021
-
[23]
U-net: Convolutional networks for biomedical image segmen- tation,
Ronneberger, O., Fischer, P., and Brox, T., “U-net: Convolutional networks for biomedical image segmen- tation,” in [Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III...
2015
-
[24]
Vision mamba for classification of breast ultrasound images,
Nasiri-Sarvi, A., Hosseini, M. S., and Rivaz, H., “Vision mamba for classification of breast ultrasound images,” MICCAI Deep Breath Workshop(2024)
2024
-
[25]
Automated breast ultrasound lesions detection using convolutional neural networks,
Yap, M. H., Pons, G., Marti, J., Ganau, S., Sentis, M., Zwiggelaar, R., Davison, A. K., and Marti, R., “Automated breast ultrasound lesions detection using convolutional neural networks,” IEEE journal of biomedical and health informatics22(4), 1218–1226 (2017)
2017
-
[26]
Visual prompt tuning,
Jia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., and Lim, S.-N., “Visual prompt tuning,” in [ European Conference on Computer Vision], 709–727, Springer (2022)
2022
-
[27]
Segmentation of arm ultrasound images in breast cancer-related lymphedema: A database and deep learning algorithm,
Goudarzi, S., Whyte, J., Boily, M., Towers, A., Kilgour, R. D., and Rivaz, H., “Segmentation of arm ultrasound images in breast cancer-related lymphedema: A database and deep learning algorithm,” IEEE Transactions on Biomedical Engineering70(9), 2552–2563 (2023)
2023
-
[28]
Registration of pre-and postresection ultrasound volumes with noncorresponding regions in neurosurgery,
Zhou, H. and Rivaz, H., “Registration of pre-and postresection ultrasound volumes with noncorresponding regions in neurosurgery,” IEEE journal of biomedical and health informatics20(5), 1240–1249 (2016)
2016
-
[29]
Cinepro: Robust training of foundation models for cancer detection in prostate ultrasound cineloops,
Harmanani, M., Jamzad, A., To, M. N. N., Wilson, P. F., Guo, Z., Fooladgar, F., Sojoudi, S., Gilany, M., Chang, S., Black, P., et al., “Cinepro: Robust training of foundation models for cancer detection in prostate ultrasound cineloops,” arXiv preprint arXiv:2501.12331(2025)
2025 arXiv
-
[30]
Active inference and deep generative modeling for cognitive ultrasound,
Van Sloun, R. J., “Active inference and deep generative modeling for cognitive ultrasound,” IEEE Trans- actions on Ultrasonics, Ferroelectrics, and Frequency Control(2024)
2024
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.