GANs 'N Lungs: improving pneumonia prediction

Elena Ericheva; Ivan Drokin; Tatiana Malygina

GANs 'N Lungs: improving pneumonia prediction

T0 review · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding CycleGAN-generated chest X-rays to a pneumonia training set raised validation ROC AUC from 0.9745 to 0.9929 in a single experiment.

desk verdict A plausible but under-controlled extended abstract: CycleGAN augmentation raises pneumonia AUC on a CheXNeXt subset, but the gain isn't yet attributable to CycleGAN because there's no balanced-data baseline, no error bars, and no size-matched control. read the letter →

arxiv 1908.00433 v1 pith:QTTKKBKH submitted 2019-08-01 eess.IV cs.CV

Tatiana Malygina , Elena Ericheva , Ivan Drokin This is my paper · ORCID

classification eess.IVcs.CV

keywords datasetimprovemethodpneumoniaaccuracyachieveaugmentationbalanced

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Pneumonia is hard to detect automatically when most training X-rays are normal, because the classifier sees far more healthy images than sick ones. The authors test a way to fix this imbalance: use a CycleGAN, a type of neural network, to convert healthy X-rays into synthetic pneumonia-like X-rays, and convert pneumonia X-rays into synthetic healthy ones. Each original image gets a synthetic partner from the opposite class, so the final training set is perfectly balanced. They then train a DenseNet-121 classifier on the original images plus the synthetic ones. On a subset of the CheXNeXt chest X-ray dataset, they report that this augmentation raised the ROC area under the curve from 0.9745 to 0.9929. They also report a smaller improvement in precision-recall. The improvement holds even when the CycleGAN is trained on the same training data as the classifier, not on extra images. The main caveat is that the comparison is not fully controlled. The augmented training set contains twice as many images as the original, so the gain might come from having more training data rather than from the synthetic images specifically. The paper does not compare against simpler techniques like duplicating existing images, random oversampling, or class-weighted loss. Measurements are reported as single numbers without confidence intervals, and the authors state that the synthetic images have not yet been validated by radiologists. So the method is promising, but the exact contribution of the GAN is not yet isolated.

Extended reading notes

Core claim

The central claim is that data augmentation with CycleGAN improves pneumonia binary classification accuracy even when the generative network is trained on the same training dataset. Specifically, the paper reports ROC AUC increasing from 0.9745 to 0.9929 (or 0.9939 with an additional dataset for CycleGAN pretraining) and PR AUC increasing from 0.9580 to 0.9865 on validation data. If correct, the method is a practical way to balance imbalanced medical imaging datasets using generated images.

Load-bearing premise

The generated images G0→1(x) and G1→0(x) are valid examples of the opposite class, meaning a synthetic 'pneumonia' image actually contains pneumonia features and a synthetic 'no abnormality' image is truly clean. The authors do not verify this; they state in the Conclusion that 'for non-radiologist it might be unclear if generated images belong to pneumonia or no pneumonia class' and that they plan to test the images against radiologists. If the labels of generated images are wrong, the improved classifier may be learning artifacts, making the measured AUC gain unreliable.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a short MIDL extended abstract reporting that CycleGAN-based oversampling lifts pneumonia classification AUC from 0.9745 to 0.9929 on a CheXNeXt subset, with a further small bump if the CycleGAN is pretrained on an additional dataset. The genuinely new piece is the same-dataset experiment—training the generator on the same training fold and still seeing an improvement—which goes a bit beyond the prior liver-lesion GAN augmentation work.

Credit where it's due: the evaluation uses held-out validation data, reports both ROC and PR AUC, and the authors are honest about the fact that generated images have not been verified by radiologists and may be visually ambiguous. Running the whole thing on a single GTX 1070 is a practical selling point, not a weakness. The class activation maps are illustrative rather than proof, but they don't oversell them.

The soft spots are concentrated in attribution. The augmented training set is twice the size and perfectly balanced; the comparison is against the original imbalanced set. Without a control that restores balance and dataset size with simple mechanisms—random oversampling, flips/rotations, or class weights—you cannot tell whether CycleGAN is doing the work or just providing more examples. There are no error bars and no repeated runs, so a 0.018 AUC difference might be stochastic. The stress-test note is right: the radiologist-validity concern is secondary. Even if the synthetic labels are imperfect, the validation metric is computed on real images, so the gain could be regularization; the missing baselines are the load-bearing weakness.

One minor but annoying issue: the experiments section says the base dataset is "pure CXR14" while the introduction says a CheXNeXt subsample. That looks like a typo, but it should be corrected because the datasets are related but not identical.

Overall, this is a reasonable workshop-level empirical note, not a rigorous demonstration. The central mechanism is unproven, but the direction is sensible and the paper is honest about its limits. I would not cite it as strong evidence in the next year, but it might be worth bringing to a reading group as an example of how easy it is to overclaim in GAN-augmentation papers.

Recommendation: send to peer review as an extended abstract. The authors should be asked to add a random-oversampling baseline and repeated runs; that is enough to make the claim much stronger.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new theoretical constructs; it relies on standard deep learning components. The central claim rests on several unreported experimental choices (hyperparameters, subsample size) and on the unverified clinical validity of the generated images. There are no invented entities.

free parameters (3)

DenseNet-121 training hyperparameters
Learning rate, batch size, epochs, optimizer settings, and regularization were not reported; these choices directly affect the measured validation AUC and are free in the presented experiment.
CycleGAN training hyperparameters
Learning rate, cycle-consistency weight, number of iterations, and image resolution for the generators were not reported; these affect the quality and style of the generated images.
CheXNeXt subsample size and class ratio
The authors selected images with pneumonia or no abnormalities from CheXNeXt but did not specify the number of images in each class or the imbalance ratio; task difficulty and the effect of balancing depend on this choice.

assumptions (3)

domain assumption The CycleGAN-generated images preserve the target class label, i.e., a generated 'pneumonia' image contains actual pneumonia-like features.
The classifier is trained using these labels; the paper does not validate the images with radiologists and explicitly lists this as future work in the Conclusion.
domain assumption The reported train/validation split is preserved and no validation data is used to train the CycleGAN or the classifier.
The paper states the original split is preserved, but does not describe how the CycleGAN training was constrained to the training subset; if validation images leaked in, the AUC comparison would be invalid.
standard math The standard implementations of DenseNet-121 and CycleGAN perform as expected without unusual configuration details.
No code or hyperparameters are provided, so the reproducibility of the result assumes common default settings for both architectures.

how reviews work

0 comments

Cite this review

Pith. "Pith review of GANs 'N Lungs: improving pneumonia prediction." pith.science (2026). https://pith.science/paper/QTTKKBKH

@misc{pith2026190800433,
  author       = {Pith},
  title        = {Pith review of: GANs 'N Lungs: improving pneumonia prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QTTKKBKH}},
  note         = {Machine review of arXiv:1908.00433}
}

read the original abstract

We propose a novel method to improve deep learning model performance on highly-imbalanced tasks. The proposed method is based on CycleGAN to achieve balanced dataset. We show that data augmentation with GAN helps to improve accuracy of pneumonia binary classification task even if the generative network was trained on the same training dataset.

Figures

Figures reproduced from arXiv: 1908.00433 by the authors.

**Figure 1.** Picture shows Original images from CheXnext and their CycleGAN-generated pairs. Difference between Original and Generated images is shown in Difference row [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

7 extracted references · 7 canonical work pages

[1]

Learning deep features for discriminative localization

Bolei Zhou et al. Learning deep features for discriminative localization. In IEEE CVPR, 2016

work page 2016
[2]

Kermany et al

Daniel S. Kermany et al. Identifying medical diagnoses and treatable diseases by image- based deep learning. Cell, 172(5):1122–1131.e9, February 2018

work page 2018
[3]

Huang et al

G. Huang et al. Densely connected convolutional networks. In IEEE CVPR , pages 2261–2269, July 2017

work page 2017
[4]

Unpaired image-to-image translation using cycle-consistent adver- sarial networks

Jun-Yan Zhu et al. Unpaired image-to-image translation using cycle-consistent adver- sarial networks. In IEEE ICCV, 2017

work page 2017
[5]

GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification

Maayan Frid-Adar et al. GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classiﬁcation. CoRR, abs/1803.01229, 2018. 3 Malygina Ericheva Drokin

work page Pith review arXiv 2018
[6]

Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists

Pranav Rajpurkar et al. Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLOS Medicine, 15 (11):1–17, 11 2018

work page 2018
[7]

ChestX-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classiﬁcation and localization of common thorax diseases

Xiaosong Wang et al. ChestX-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classiﬁcation and localization of common thorax diseases. IEEE CVPR, pages 2097–2106, 2017. 4

work page 2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Learning deep features for discriminative localization

Bolei Zhou et al. Learning deep features for discriminative localization. In IEEE CVPR, 2016

work page 2016

[2] [2]

Kermany et al

Daniel S. Kermany et al. Identifying medical diagnoses and treatable diseases by image- based deep learning. Cell, 172(5):1122–1131.e9, February 2018

work page 2018

[3] [3]

Huang et al

G. Huang et al. Densely connected convolutional networks. In IEEE CVPR , pages 2261–2269, July 2017

work page 2017

[4] [4]

Unpaired image-to-image translation using cycle-consistent adver- sarial networks

Jun-Yan Zhu et al. Unpaired image-to-image translation using cycle-consistent adver- sarial networks. In IEEE ICCV, 2017

work page 2017

[5] [5]

GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification

Maayan Frid-Adar et al. GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classiﬁcation. CoRR, abs/1803.01229, 2018. 3 Malygina Ericheva Drokin

work page Pith review arXiv 2018

[6] [6]

Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists

Pranav Rajpurkar et al. Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLOS Medicine, 15 (11):1–17, 11 2018

work page 2018

[7] [7]

ChestX-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classiﬁcation and localization of common thorax diseases

Xiaosong Wang et al. ChestX-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classiﬁcation and localization of common thorax diseases. IEEE CVPR, pages 2097–2106, 2017. 4

work page 2017