{"id":"fd811a2b-00a1-42f6-9c56-e453e570e5ec","arxiv_id":"2501.15248","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Pretraining classifiers on diffusion-generated synthetic fetal ultrasound images and fine-tuning on real images improves classification accuracy across five architectures, mostly for underrepresented classes.","lead":"This paper generates 60,000 synthetic fetal ultrasound images with a diffusion model and shows that pretraining image classifiers on them, then fine-tuning on real images, improves classification accuracy compared with training on real images alone. It is a data-scarcity workaround for medical imaging, where labeled ultrasound data is expensive and hard to obtain.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Attribution to diffusion-generated images is not established: Table 12 shows traditional augmentation pretraining matches or beats diffusion pretraining for 4/5 classifiers, and the main comparison confounds data source with training budget and test-set checkpoint selection.","rationale":"The reader identified the correct weak point: the main comparison varies data source, corpus size, and training budget simultaneously. I agree, and I think Table 12 makes the problem sharper than the reader stated. Under the natural reading of the row labels, traditional augmentation pretraining plus fine-tuning beats diffusion pretraining plus fine-tuning for four of five models. That is not merely a missing control; it is evidence against the diffusion-specific attribution. I am not claiming the result is fraudulent or that diffusion can never help; the table's row semantics are ambiguous and no code or data is released, so the contradiction cannot be checked. A controlled reproduction with validation-based checkpoint selection and multiple seeds is the decisive experiment. Because the current manuscript does not allow that check, I would move the verdict from CONDITIONAL to UNVERDICTED rather than accept the causal claim as supported.","tokens_in":15408,"tokens_out":9736,"duration_ms":89521,"concrete_test":"Reproduce Table 12 rows 2 and 4 with identical training budgets (200+200 epochs), identical optimizer settings, and checkpoints chosen on a held-out validation split rather than the test set, across five seeds. If 'Pretraining (30000 traditional augmentation) + Fine-Tuning (9923 real)' is again at least as good as 'Pretraining (30000 synthetic images) + Fine-Tuning (9923 real)' for at least four of five classifiers, the paper's causal attribution to diffusion-generated data fails. Also request the authors' exact row definitions and code to rule out a mislabeled table.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The paper's central causal claim—that diffusion-generated synthetic fetal ultrasound images are what improve downstream classifiers—is undermined by an uncontrolled comparison and by the paper's own ablation. The headline result (Tables 5-9) compares 'Real Data Only' (200 epochs on 9,923 images) with 'Pretraining + Fine-Tuning' (200 epochs on 30,000/60,000 synthetic images plus 200 more epochs on real data), so data source, dataset size, and optimization budget all change at once. The ablation in Table 12, read literally, is stronger evidence against the diffusion-specific interpretation: the row 'Pretraining (30000 traditional augmentation) + Fine-Tuning (9923 real)' exceeds the row 'Pretraining (30000 synthetic images) + Fine-Tuning (9923 real)' for four of five classifiers (ResNet50 93.8 vs 92.5; DenseNet169 93.7 vs 92.7; Swin_t 92.7 vs 91.5; MedMamba 93.0 vs 92.1; ViT_b_32 88.2 vs 89.6). If those rows are comparable, the benefit is produced by pretraining on a larger, augmented corpus, not specifically by diffusion-model-generated content. Section 4.3 compounds the problem by saving the best test accuracy during training, selecting checkpoints on the test set with no seeds or error bars, so reported gains are upper bounds. The synthetic images are announced as 'will be made publicly available,' but no code or data is currently released, preventing verification of the conflicting ablation rows.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes using a classifier-guided diffusion model trained on the FETAL_PLANES_DB training split to generate 60,000 synthetic fetal ultrasound images (30,000 in each of two rounds). The authors then pretrain five classifiers (ResNet50, DenseNet169, ViT_b_32, Swin_t, MedMamba) on the synthetic images and fine-tune them on the real training images, reporting that this two-stage procedure yields higher test accuracy than training on real images alone. The paper also includes visual and FID-based quality assessment of the synthetic images, confusion-matrix analyses, a majority-vote ensemble, comparisons to two prior fetal-plane classification methods, and three ablation studies. The central claim is that diffusion-generated synthetic images improve downstream classification accuracy relative to training with real data only.","tokens_in":15714,"tokens_out":2949,"duration_ms":25559,"significance":"If the central claim were established, the work would provide a practical recipe for mitigating data scarcity in fetal ultrasound classification and would contribute a large public synthetic dataset as a community resource. The paper has some genuine strengths: it evaluates five diverse classifier architectures, reports consistent directional improvements across them, and releases (or announces intent to release) the synthetic images. However, as it stands, the attribution of the gains to diffusion-model-generated content is not supported because the main comparison confounds data source with dataset size and optimization budget, and the paper's own ablation (Table 12) shows that pretraining on traditional augmentations matches or exceeds pretraining on diffusion images for four of five classifiers. The evaluation protocol also selects checkpoints on the test set and provides no error bars, so the reported numbers should be read as optimistic upper bounds. With additional controls, matched training budgets, and a corrected evaluation protocol, the underlying question is worth pursuing, but the current manuscript does not yet justify its headline conclusion.","major_comments":[{"comment":"The main comparison confounds the data source with the number of training iterations and the pretraining corpus size. 'Real Data Only' trains for 200 epochs on 9,923 real images, while 'Pretraining + Fine-Tuning' trains for 200 epochs on 30,000 or 60,000 synthetic images and then fine-tunes for 200 more epochs on the real data. The observed test-accuracy gains could therefore come from the additional optimization budget or the larger pretraining set rather than from the diffusion-model-specific distribution. A matched control, such as training 'Real Data Only' for 400 epochs, or pretraining on 30,000/60,000 traditional augmentations of the real images followed by the same fine-tuning and checkpoint-selection protocol, is needed before one can attribute the improvement to diffusion-generated content.","section":"§5.2.1, Tables 5–9"},{"comment":"The ablation intended to control for traditional augmentation actually undermines the diffusion-specific claim. In the rows 'Pretraining (30000 traditional augmentation) + Fine-Tuning (9923 real)' versus 'Pretraining (30000 synthetic images) + Fine-Tuning (9923 real)', the traditional-augmentation pretraining achieves higher test accuracy for four of the five classifiers (ResNet50 93.8 vs. 92.5; DenseNet169 93.7 vs. 92.7; Swin_t 92.7 vs. 91.5; MedMamba 93.0 vs. 92.1), with only ViT_b_32 showing the opposite (88.2 vs. 89.6). This pattern is stronger evidence that the benefit arises from pretraining on a larger, more varied corpus than from the specific synthetic distribution emitted by the diffusion model. The paper's stated interpretation of Table 12 is therefore not supported by its own numbers, and the central claim should be revised or the experiments redesigned.","section":"§5.3.1, Table 12"},{"comment":"The evaluation protocol selects checkpoints using the test set: 'we saved the best training and test accuracy during the training procedure.' This is a test-set leak, since the reported test accuracy is the maximum over 200 epochs rather than the accuracy of a fixed model chosen without looking at the test labels. No validation set or early-stopping criterion is described, and no standard errors or multiple seeds are reported, so the magnitude of the claimed improvements (e.g., 90.8% vs. 75.5% for ViT_b_32 in Table 5) is not trustworthy as an unbiased estimate. The authors should hold out a validation split for checkpoint selection and report mean and standard deviation over at least three independent runs.","section":"§4.3"},{"comment":"Table 10 reports test accuracies that are inconsistent with the corresponding entries in Tables 5–9. For example, Table 10 lists ResNet50 at 93.94%, DenseNet169 at 94.07%, Swin_t at 93.10%, MedMamba at 93.50%, and Vit_b at 90.15%, while the closest rows in Tables 5–9 ('Pretraining (60000 synthetic images) + Fine-Tuning') give 93.2%, 92.8%, 93.1%, 92.4%, and 90.8%, respectively. The caption of Table 10 says 'the Final Training Strategy: Pretraining with Combined Synthetic Data and Fine-Tuning,' but the relationship between this strategy and the 30k/60k pretraining setups in Tables 5–9 is not specified. This discrepancy makes the main results irreproducible as reported and needs to be resolved with a clear definition of which experimental configuration generated each table.","section":"§5.2.3, Table 10 vs. Tables 5–9"},{"comment":"The 'good'/'bad' split used for the FID analysis and for the downstream comparison in Table 4 is based on an ad hoc procedure: initially 5,000 'good' and 'bad' images in total were manually selected, then a binary classifier was trained to label the remaining images. No details are given for this classifier (architecture, training set size per class, accuracy, or whether it was trained on the same classes), and it is unclear how many images of each type were used in the downstream pretraining of Table 4. Without these details, the FID scores in Table 3 and the comparison in Table 4 cannot be interpreted quantitatively, and the claim that both 'good' and 'bad' images contribute positively is not properly supported.","section":"§5.1.2–§5.1.3"}],"minor_comments":[{"comment":"The text states that the augmented dataset contains 12,700 images, but Table 1 lists a total of 12,400 images in FETAL_PLANES_DB; this inconsistency should be corrected.","section":"§3.4"},{"comment":"The paper repeatedly refers to 'six different classifiers' (in the abstract contributions and conclusion), but only five classifiers (ResNet50, DenseNet169, ViT_b_32, Swin_t, MedMamba) are used in the main experiments; the sixth appears only in the ensemble vote in Table 11, and the text should be clarified.","section":"§1 and §6"},{"comment":"There are typographical issues such as 'V AEs' (should be 'VAEs') and inconsistent spacing in the related-work section; a careful proofread is needed.","section":"§2.2"},{"comment":"The 'half U-Net' used as the classifier for classifier guidance is described only briefly; the architecture, number of parameters, and training details (including how the gradient is injected during sampling) are not specified, which prevents replication of the generation pipeline.","section":"§3.2 and §4.4"},{"comment":"The comparisons to Krishna and Kokil [2024a] and Krishna and Kokil [2023] report lower accuracies (94.64% and 93.86%) than the original papers (95.69% and 95.5%), but it is not stated whether the same train/test split, image size, and checkpoint selection were used; this makes the comparison difficult to assess.","section":"§5.2.5"},{"comment":"The paper states that the synthetic images 'will be made publicly available' but provides no link or code release; given that the main contribution is the dataset, the authors should release the images, the generation code, and the training scripts at publication time.","section":"Availability"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for a computer-vision or medical-imaging venue, but the central claim is currently not established because of the confounded comparison and the contradictory ablation in Table 12. The inconsistency between Table 10 and Tables 5–9 is particularly concerning and should be addressed head-on. The announced release of the synthetic dataset is a valuable resource, and with a properly controlled ablation, matched training budgets, and error bars, the paper could become a useful empirical study. If the authors cannot show an advantage over traditional augmentation pretraining, the framing should be changed substantially."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's central claim—that diffusion-generated synthetic images are what improve downstream classifiers—is not established. The main comparison changes data source and training budget at once, and the paper's own Table 12 shows traditional augmentation pretraining matching or beating diffusion pretraining for four of five classifiers. That is the strongest evidence against the diffusion-specific interpretation, and the paper does not address it.\n\nWhat is genuinely new: first application of classifier-guided diffusion to FETAL_PLANES_DB, a 60,000-image synthetic set (though not yet released), and the two-stage pretrain-fine-tune recipe is tested across five architectures. The direction is consistent: every model improves over real-data-only. The confusion-matrix analysis showing the biggest gains on the rare classes (abdomen, femur) is the most credible part of the paper.\n\nSoft spots, in order:\n\n1. Attribution. The baseline 'Real Data Only' gets 200 epochs on ~10k images; the proposed method gets 200 epochs on 30k/60k synthetic plus 200 more on real. The gain could come from extra optimization steps or a larger corpus, not from diffusion. Table 12, which compares equal-size pretraining corpora, shows traditional augmentation beats diffusion for ResNet50, DenseNet169, Swin_t, and MedMamba. Read literally, this says the benefit is from pretraining on a bigger, more varied set, not from diffusion-generated content. The text's claim that diffusion 'consistently outperforms' the traditional baseline is not what the numbers show.\n\n2. Protocol. Section 4.3 says they 'saved the best training and test accuracy during the training procedure.' Selecting checkpoints on the test set means the reported numbers are upper bounds, not honest estimates. No seeds, no error bars, so the ~1-point gains are within noise.\n\n3. Internal inconsistency. Table 10 reports test accuracies (93.94% for ResNet50, etc.) that do not match Tables 5–9 (93.2% for ResNet50 with 60k pretraining). The caption says 'final training strategy' but no protocol in the main tables produces those numbers.\n\n4. Reproducibility. No code or data released. The synthetic images are announced as 'will be made publicly available,' which is not the same as available.\n\nWho this is for: a reader interested in quick recipes for ultrasound data augmentation might get ideas, but should not cite the numbers as reliable effect sizes.\n\nRecommendation: deserves a serious referee. The workload is manageable and the questions are fixable. Ask for validation-based model selection, multiple seeds, a matched-budget baseline (same total epochs), and a direct test of the diffusion-specific claim against traditional augmentation. With those changes, the paper could be a solid incremental contribution.","headline":"A useful demonstration of diffusion-based augmentation for fetal ultrasound, but the attribution of gains to diffusion itself is not supported by the authors' own ablation.","tokens_in":16250,"tokens_out":2329,"would_cite":false,"duration_ms":20535,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fetal plane classifiers trained first on diffusion-generated ultrasound images and then fine-tuned on real images outperform classifiers trained on real images alone.","keywords":["diffusion models","data augmentation","fetal ultrasound","image classification","synthetic medical images","pretraining and fine-tuning","data scarcity","FETAL_PLANES_DB"],"falsifier":"A control experiment could settle the question: train each classifier for 400 epochs on the 9,923 real images alone to match the total training budget, or pretrain on 60,000 traditionally augmented real images (rotations, flips, crops) before the same fine-tuning; if either control matches or beats the reported pretrain-plus-fine-tune accuracy, the attribution of the gain to diffusion-generated synthetic data would be refuted.","tokens_in":15209,"feed_emoji":"👶","tokens_out":7259,"duration_ms":55913,"temperature":0.7,"pith_summary":"The paper sets out to show that diffusion-model-generated synthetic fetal ultrasound images can serve as a training resource that improves classifier accuracy beyond what real images alone provide. The authors generate 60,000 class-conditional synthetic images from the FETAL_PLANES_DB training set, pretrain five different classifiers on a 30,000- or 60,000-image synthetic corpus, and then fine-tune on the real images. Across every classifier they test, the pretrain-then-fine-tune recipe yields higher test accuracy than training only on the real data, with the best single model reaching 93.2% and a majority-vote ensemble reaching 94.91%. The gains are most visible for the fetal abdomen and fetal femur classes, which have the fewest real training examples.","feed_headline":"Synthetic ultrasounds beat real-only training in fetal scans","feed_subtitle":"Pretraining on 60,000 generated images, then fine-tuning on real data, lifts accuracy for all five classifiers.","key_machinery":"The machinery that carries the argument is a classifier-guided diffusion model: a denoising diffusion probabilistic model whose reverse denoising process is steered by a half-U-Net classifier trained alongside it, so that sampling conditioned on a label produces images of the corresponding fetal plane. Two rounds of generation produce 10,000 synthetic images per class (60,000 total), and the downstream gain comes from the two-stage training recipe that first pretrains each classifier on that synthetic corpus and then fully fine-tunes it on the 9,923 real training images. The consistency of the gain across five architecturally diverse classifiers, and across two synthetic corpus sizes, is what the paper offers as evidence that the synthetic data, not any single classifier, is doing the work.","core_discovery":"The paper's central claim is that a classifier-guided diffusion model trained on the real fetal ultrasound training split can generate enough class-labeled synthetic images that pretraining a downstream classifier on them, followed by fine-tuning on the real data, outperforms training on the real data alone. This claim is tested with ResNet50, DenseNet169, ViT_b_32, Swin_t, and MedMamba on the FETAL_PLANES_DB benchmark. In the main comparison, the 60,000-synthetic-image pretraining plus fine-tuning row beats the Real-Data-Only row for every classifier (for example, ResNet50 improves from 92.0% to 93.2% test accuracy). The authors also report that increasing the synthetic corpus from 30,000 to 60,000 images consistently improves the final accuracy, and that the benefit is concentrated in classes with the smallest real training sets.","pith_inferences":["The paper's own ablation shows that removing classifier guidance slightly improves downstream accuracy, so a simpler unguided diffusion model may be a cheaper and equally effective generator for this application.","Because the main comparison does not equalize total optimization steps between the real-data baseline and the synthetic-pretraining recipe, the diffusion-specific contribution remains entangled with extra training effort; a matched-budget control would likely change how the headline result is read.","The same pretrain-on-synthetic, fine-tune-on-real strategy should transfer to other ultrasound tasks such as biometry plane identification or anomaly detection, and to other modalities like MRI and CT where annotated data are similarly scarce."],"forward_implications":["If the claim holds, any medical imaging domain with a small annotated set can raise classifier accuracy by pretraining on a large diffusion-generated corpus and then fine-tuning on the real images, without acquiring new real labels.","Because the biggest gains appear in classes with the fewest real examples, the recipe offers a targeted remedy for class imbalance as well as overall data scarcity.","The reported trend that 60,000 synthetic images beat 30,000 implies that generating more synthetic data may keep improving downstream accuracy at least up to the tested scale.","The authors' plan to release the 60,000-image synthetic dataset would give other researchers a ready-made pretraining resource for fetal ultrasound classification."],"supporting_citations":[{"why":"Supplies the classifier-guided diffusion method used to generate class-conditioned synthetic images.","marker":"[Dhariwal and Nichol, 2021]"},{"why":"Provides the denoising diffusion probabilistic model framework the generator is built on.","marker":"[Ho et al., 2020]"},{"why":"Contributes the FETAL_PLANES_DB dataset of 12,400 real fetal ultrasound images used for training and testing.","marker":"[Burgos-Artizzu et al., 2020]"},{"why":"Stacked-ensemble baseline whose reported accuracy the paper re-implements and compares against.","marker":"[Krishna and Kokil, 2024a]"},{"why":"Deep-feature-integration baseline also re-implemented for comparison.","marker":"[Krishna and Kokil, 2023]"},{"why":"Establishes the precedent of GAN-generated medical images for classifier augmentation, the approach this paper extends to diffusion models.","marker":"[Frid-Adar et al., 2018]"},{"why":"Closest prior work using a diffusion model to generate liver ultrasound images for classification, motivating the fetal application.","marker":"[Hardy et al., 2023]"}],"fun_headline_variants":["Diffusion-generated ultrasounds boost fetal scan accuracy","Synthetic data from diffusion models improves fetal classification","Pretraining on fake ultrasounds tops real-only in fetal scans","Fetal plane accuracy rises with diffusion-based augmentation","Diffusion models create training data that beats real images"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's conclusion depends on the assumption that the accuracy gain comes from the diffusion-generated images themselves and not from the fact that the proposed training procedure simply runs many more optimization steps on a much larger pretraining corpus than the real-data-only baseline.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion-generated ultrasounds boost fetal scan accuracy","Synthetic data from diffusion models improves fetal classification","Pretraining on fake ultrasounds tops real-only in fetal scans","Fetal plane accuracy rises with diffusion-based augmentation","Diffusion models create training data that beats real images"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000695,"raw_usage":{"total_tokens":3085,"prompt_tokens":832,"completion_tokens":2253,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":448,"completion_tokens_details":{"reasoning_tokens":2178}},"tokens_in":448,"tokens_out":2253,"duration_ms":14150,"temperature":1.0,"reasoning_tokens":2178,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:27:47.366136+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A control experiment could settle the question: train each classifier for 400 epochs on the 9,923 real images alone to match the total training budget, or pretrain on 60,000 traditionally augmented real images (rotations, flips, crops) before the same fine-tuning; if either control matches or beats the reported pretrain-plus-fine-tune accuracy, the attribution of the gain to diffusion-generated synthetic data would be refuted.","supporting_citations":[{"cited_title":"Evaluation of deep convolutional neural networks for automatic classification of common maternal fetal ultrasound planes","cited_arxiv_id":null,"evidence_quote":"Contributes the FETAL_PLANES_DB dataset of 12,400 real fetal ultrasound images used for training and testing."},{"cited_title":"Automated classification of common maternal fetal ultrasound planes using multi-layer perceptron with deep feature integration","cited_arxiv_id":null,"evidence_quote":"Deep-feature-integration baseline also re-implemented for comparison."},{"cited_title":"Gan-based synthetic medical image augmentation for increased cnn performance in liver lesion classification","cited_arxiv_id":null,"evidence_quote":"Establishes the precedent of GAN-generated medical images for classifier augmentation, the approach this paper extends to diffusion models."},{"cited_title":"Improving nonalcoholic fatty liver disease classification performance with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Closest prior work using a diffusion model to generate liver ultrasound images for classification, motivating the fetal application."}],"review_version":1}