REVIEW 3 major objections 6 minor 20 references
CompDiff: Hierarchical Compositional Diffusion for Fair and Zero-Shot Intersectional Medical Image Generation
T0 review · 3 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read CompDiff shows demographic conditioning can be factorized hierarchically so unseen intersections compose from seen attributes.
desk verdict Representation-level compositional conditioning is a genuinely new idea with solid scaffolding, but the zero-shot evidence is statistically under-specified and one ablation has an internal inconsistency. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the Hierarchical Conditioner Network (HCN), which decomposes demographic conditioning into single-attribute embeddings ('grandparents'), pairwise interaction MLPs ('parents'), and a full-composition MLP ('child') that yields the demographic token c. The token is concatenated with CLIP text embeddings to form the cross-attention context for a Stable Diffusion UNet. The design is regularized by a compositional consistency term Lcomp = 1 - cos(hdemo, e_age + e_sex + e_race), a KL term on the variational latent, and auxiliary classification applied to the projected token. The HCN's structured factorization encourages parameter sharing across subgroups and is what allows
What would settle it
Generate or collect a test set where each demographic intersection has hundreds of real images, remove several intersections from training, and compare CompDiff against a flat encoder and a reweighting baseline with confidence intervals on subgroup FID. If the held-out intersections do not show a statistically significant FID advantage—or if an ablation replacing the hierarchy with a flat MLP of equal capacity matches CompDiff—the compositional-generalization claim is falsified. A second probe: train on synthetic data with known ground-truth attribute factors and check whether generated images
Extended reading notes
Core claim
This paper claims that demographic identity in medical images is compositional: a diffusion model can generalize to unseen demographic intersections if conditioning is factored hierarchically. CompDiff's Hierarchical Conditioner Network (HCN) embeds age, sex, race separately, learns pairwise interactions, and fuses them into a demographic token appended to CLIP text embeddings for cross-attention. Auxiliary classification on the projected token preserves demographic information; a variational latent and a cosine anchor to the additive baseline regularize training. On five held-out intersections with zero training examples, CompDiff improves FID by up to 21%, whereas FairDiffusion, which rewe
Load-bearing premise
Zero-shot generalization is measured on five held-out intersections whose real test samples number only tens of images, so the subgroup FID values may not be stable enough to reliably separate methods; underneath that sits the premise that pairwise interactions trained on seen pairs compose correctly to unseen triples.
Editorial extensions
If this is right
- If CompDiff's central claim holds, loss-reweighting methods like FairDiffusion cannot be the complete answer to the imbalanced generator problem, because they cannot synthesize training signal for combinations absent from the data.
- Rare demographic intersections can be generated with acceptable quality even when no training examples exist, which opens a path to augmenting clinical datasets for underrepresented groups.
- The architecture of demographic conditioning—not just the loss—affects downstream classifier fairness; classifiers trained on CompDiff data show higher AUROC and lower underdiagnosis or equalized-odds differences.
- Because HCN adds only 0.19% trainable parameters and requires no change to sampling, the approach is a cheap drop-in replacement for the conditioning head of a latent diffusion model.
Reading between the lines
- Editorial: If composition works for demographics, the same hierarchical factorization could be applied to other structured clinical attributes (e.g., pathology, severity, body region), potentially improving generation for rare disease combinations.
- Editorial: A direct test would compare CompDiff and flat encoders on a synthetic dataset with known generative factors to see whether the hierarchy genuinely recovers compositionality or simply provides extra capacity.
- Editorial: The paper's reliance on pairwise interactions raises a question: will triple or higher-order interactions degrade substantially when all three attributes are unseen together? Future graph-based interaction models might capture these better.
- Editorial: If zero-shot compositional generation is confirmed, it reduces the need to gather rare demographic data for training, but clinical expert evaluation (not just automated metrics) would be needed before synthetic images are used in practice.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. CompDiff proposes a representation-level solution to demographic imbalance in medical image diffusion models. Instead of reweighting losses, it introduces a Hierarchical Conditioner Network (HCN) that decomposes demographic conditioning into single-attribute embeddings, pairwise interactions, and a composed token, which is concatenated with CLIP text embeddings as cross-attention context. The training objective combines the diffusion loss with a compositional-consistency loss, a KL term on the demographic latent, and an auxiliary classification loss on the final conditioning token. The paper reports improved FID, ES-FID, zero-shot FID on five held-out demographic intersections, and downstream classifier fairness on MIMIC-CXR and FairGenMed, with ablations supporting the hierarchical design and the auxiliary loss placement.
Significance. The central idea is timely and, if correct, significant: it reframes fairness interventions from optimization-level reweighting to representational factorization and provides initial evidence that unseen demographic intersections can be generated in a zero-shot manner. The experimental scaffolding is strong in several respects: three-seed reporting, held-out generated test sets, external pretrained classifiers, TSTR downstream evaluation, and systematic ablations of both architecture and loss terms. The released code is a further strength. The main reservation is not the architecture but the statistical basis of the headline zero-shot claim, which currently rests on a single table without uncertainty estimates. With that evidence properly grounded, the paper would be a solid contribution to fair medical image generation.
major comments (3)
- [§3.3, Table 3] The protocol sentence in §3.3 states that all results are computed on held-out generated test sets using three generation seeds and that mean and standard deviation are reported, but Table 3 reports only point estimates for each method and intersection. No per-subgroup real-sample size is given. The held-out intersections were chosen by rarity; the rarest rows in Table 2 are 0.5% of training, so the real reference sets are likely on the order of tens of images. Subgroup FID at this scale is high-variance and depends on the real-image count. As a result, the claimed up-to-21% improvement (e.g., 18–40 M/A: 161.3→127.6) and the FairDiffusion failures (247.2 vs 210.7 for 80+ F/A; 265.5 vs 208.1 for 80+ M/A) cannot be separated from noise as reported. This is the only direct evidence for the zero-shot claim. Please provide per-subgroup n, mean(std) over seeds, and ideally bootstrap confidence
- [§2.3, Eq. (5), Tables 1/5] The auxiliary classification loss Laux is applied to the final token c using true age/sex/race labels. Consequently, the high sex/race accuracy and low age RMSE reported in Tables 1 and 5 are partly supervised properties of the conditioning pathway, not independent evidence that HCN's compositional factorization yields demographic control. The ablation HCN(no aux) versus CompDiff shows that the loss is necessary, and the flat-versus-hierarchical comparison under identical supervision is informative. However, the paper should state explicitly that demographic controllability is partly constructed by Eq. (5), and should ideally validate the semantics of c with an independent probing task not trained jointly with the generator before using demographic accuracy as evidence for the representation-level claim.
- [§2.2, §3.3] The zero-shot hypothesis is that pairwise interactions (h_a,s, h_a,r, h_s,r) and the Lcomp anchor compose correctly for an unseen triple. Table 3 lists only the removed triples; it does not report whether the constituent pairs were present in training or how often. For example, 80+ F/A has pairs (80+, F), (80+, A), (F, A), and if any pair is also absent or extremely rare, the pairwise-to-triple composition story is not the mechanism actually tested. Please report per-held-out-group pair coverage (counts or percentages) and, if possible, include a control where seen triples of matched rarity are removed to confirm that the method's advantage is specific to true zero-shot composition rather than to rarity in general.
minor comments (6)
- [Eq. (6)] The ES-FID formula is ambiguous as typeset. If the intended expression is FID * (1 + (1/(|Ai|*FID)) * sum |FID - FID_j|) = FID + mean absolute deviation, please rewrite it to remove the apparent extra FID factor. As printed, the units are not dimensionally consistent.
- [§3.4, Table 5] The text states that increasing the compositional-consistency weight degrades FID to 104.2, but Table 5's 'Strong Lcomp' row reports FID 97.1 for λcomp=0.5. Either the 104.2 value corresponds to a different λcomp that is omitted from the table, or the text and table do not agree. Please reconcile.
- [Table 1] The fundus baseline disease AUROC is reported without a standard deviation (0.94 with no std), while all other cells in the table have mean(std). Please add the missing value.
- [§3.3] The selection of the best checkpoint based on 'validation performance across the four dimensions' is not specified. State how the four metrics were aggregated into a single criterion and confirm that no hyperparameter or checkpoint selection was performed on the test set.
- [§3.1, §3.3] The zero-shot experiment is reported only for the chest X-ray modality. If the method is presented as a general framework for medical images, either add a zero-shot evaluation on the fundus data or explicitly restrict the zero-shot claim to the modality tested.
- [§2.3] The values of λcomp, λaux, and λKL are not given in the main text. Please report them and state whether they were tuned on the validation set; this is needed for reproducibility of the ablations.
Circularity Check
No significant circularity: CompDiff's central claims are empirical comparisons against external benchmarks, not derivations equivalent to fitted inputs by construction.
full rationale
The paper's load-bearing claims—overall FID, ES-FID, zero-shot FID on held-out intersections, and TSTR AUROC—are measured against real test images using external metrics (FID, pretrained classifiers) rather than derived from the model's own training objectives in a way that would force the result. The HCN composes single-attribute and pairwise embeddings into hdemo (Eqs. 1–3), but whether this yields high-quality images for unseen triples is an empirical question answered by Table 3, not a tautology. The auxiliary classification loss (Eq. 5) does supervise the demographic token c and therefore partly induces high sex/race classification accuracy on generated images; however, the reported accuracy is evaluated with separate pretrained classifiers on images, is near-saturated for all baselines (Table 1: 98.3 vs 0.99), and is peripheral to the central fairness/zero-shot claims. There are no load-bearing self-citations (references 8, 16, 17 are external works by other groups), no imported uniqueness theorems, and no ansatz smuggled in via citation. The absence of per-subgroup sample sizes and standard deviations in Table 3 is a statistical robustness concern, not a circularity. The manuscript's own limitation statement (performance still degrades relative to well-represented groups) is consistent with an honest, non-circular evaluation.
Assumptions & free parameters
free parameters (4)
- λcomp (compositional consistency weight) =
0.1
- λaux and λKL =
not reported in text
- HCN dimensions and pairwise MLP widths =
dnode=256, dctx=1024; MLP widths unspecified
- held-out subgroup selection =
five rarest intersections
assumptions (3)
- domain assumption Demographic identity is compositional: rare intersections can be expressed as combinations of well-learned single attributes and pairwise interactions.
- ad hoc to paper The concatenated 78th demographic token is interpretable by the fine-tuned UNet cross-attention without breaking the pretrained text-conditional distribution.
- domain assumption CLIP text embeddings of clinical findings remain informative after demographic tokens are removed from the prompt.
Cite this review
Pith. "Pith review of CompDiff: Hierarchical Compositional Diffusion for Fair and Zero-Shot Intersectional Medical Image Generation." pith.science (2026). https://pith.science/paper/ATGXJRBN
@misc{pith2026260316551,
author = {Pith},
title = {Pith review of: CompDiff: Hierarchical Compositional Diffusion for Fair and Zero-Shot Intersectional Medical Image Generation},
year = {2026},
howpublished = {\url{https://pith.science/paper/ATGXJRBN}},
note = {Machine review of arXiv:2603.16551}
}
read the original abstract
Generative models are increasingly used to augment medical imaging datasets for fairer AI, yet a key assumption often goes unexamined: that generators produce equally high-quality images across demographic groups. Models trained on imbalanced data inherit these imbalances, degrading synthesis for rare subgroups and struggling with intersections absent from training: the imbalanced generator problem. Remedies such as loss reweighting operate at the optimization level and provide limited benefit when training signal is scarce or absent. We propose CompDiff, a hierarchical compositional diffusion framework that addresses this at the representation level. A dedicated Hierarchical Conditioner Network (HCN) decomposes demographic conditioning into single-attribute, pairwise, and composed representations, producing a demographic token concatenated with CLIP embeddings as cross-attention context. This structured factorization encourages parameter sharing across subgroups and supports compositional generalization to rare or unseen intersections. On chest X-rays (MIMIC-CXR) and fundus images (FairGenMed), CompDiff compares favorably against standard fine-tuning and FairDiffusion across image quality (FID 64.3 vs. 75.1), subgroup equity (ES-FID), and zero-shot intersectional generalization (up to 21% FID improvement on held-out intersections). Downstream classifiers trained on CompDiff data show improved AUROC and reduced demographic bias, suggesting that the architectural design of demographic conditioning is an important and underexplored factor in fair medical image generation. Code: https://github.com/mahmoudibrahim98/CompDiff.
Reference graph
Works this paper leans on
-
[1]
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In:Adv. Neural Inf. Process. Syst.33, 6840–6851 (2020)
2020
-
[2]
In:International Conference on Learning Representations (ICLR) (2021)
Song, Y., et al.: Solving inverse problems in medical imaging with score-based gen- erative models. In:International Conference on Learning Representations (ICLR) (2021)
2021
-
[3]
In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10684–10695 (2022)
2022
-
[4]
Chambon, P., Bluethgen, C., Langlotz, C.P., Chaudhari, A.: RoentGen: Vision- language foundation model for chest X-ray generation. arXiv:2211.12737 (2022)
arXiv 2022
-
[5]
Bluethgen, C., et al.: A vision–language foundation model for the generation of realistic chest X-ray images.Nat. Biomed. Eng.(2024)
2024
-
[6]
Moroianu, S.L., Bluethgen, C., Chambon, P., Cherti, M., Delbrouck, J.-B., Paschali, M.,Price,B.,Gichoya,J.,Jitsev,J.,Langlotz,C.P.,Chaudhari,A.S.:Improvingper- formance,robustness,andfairnessofradiographicAImodelswithfinely-controllable synthetic data. arXiv:2508.16783 (2025)
arXiv 2025
-
[7]
Med.30, 1166–1173 (2024)
Ktena, I., et al.: Generative models improve fairness of medical classifiers under distribution shifts.Nat. Med.30, 1166–1173 (2024)
2024
-
[8]
Luo, Y., et al.: FairDiffusion: Enhancing equity in latent diffusion models via fair Bayesian perturbation.Sci. Adv.11, eads4593 (2025). https://doi.org/10.1126/sciadv.ads4593
Show all 20 references
-
[9]
arXiv:2103.00020 (2021)
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning Transferable Visual Models From Natural Language Supervision. arXiv:2103.00020 (2021)
2021 arXiv
-
[10]
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In: Adv. Neural Inf. Process. Syst.30, 6629–6640 (2017)
2017
-
[11]
Mei, S., et al.: RadImageNet: An open radiologic deep learning research dataset. Radiol. Artif. Intell.4, e210315 (2022)
2022
-
[12]
In:European Conference on Computer Vision (ECCV) (2022)
Boecking, B., et al.: Making the most of text semantics to improve biomedical vision-language processing. In:European Conference on Computer Vision (ECCV) (2022)
2022
-
[13]
Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multi-scale structural similarity for im- age quality assessment. In:Proc. Asilomar Conf. Signals, Syst. Comput., vol. 2, pp. 1398–1402 (2003) 10 M. Ibrahim et al
2003
-
[14]
arXiv:2111.00595 (2021)
Cohen, J., et al.: TorchXRayVision: A library of chest X-ray datasets and models. arXiv:2111.00595 (2021)
2021 arXiv
-
[15]
Glocker, B., Jones, C., Bernhardt, M., Winzeck, S.: Algorithmic encoding of pro- tected characteristics in chest X-ray disease detection models.eBioMedicine89, 104467 (2023)
2023
-
[16]
arXiv:2310.02492 (2024)
Luo, Y., Tian, Y., Shi, M., Elze, T., Wang, M.: Fairvision: equitable deep learning for eye disease screening via fair identity scaling. arXiv:2310.02492 (2024)
2024 arXiv
-
[17]
In:International Conference on Learning Representations (ICLR)(2024)
Tian, Y., Shi, M., Luo, Y., Kouhana, A., Elze, T., Wang, M.: FairSeg: A large- scale medical image segmentation dataset for fairness learning with fair error-bound scaling. In:International Conference on Learning Representations (ICLR)(2024)
2024
-
[18]
In: Proc
Cohen, J., et al.: Age prediction from chest radiographs using deep learning. In: Proc. Mach. Learn. Res.149, 39–53 (2021)
2021
-
[19]
Seyyed-Kalantari, L., Liu, G., McDermott, M., Chen, I.Y., Ghassemi, M.: CheX- clusion: Fairness gaps in deep chest X-ray classifiers. In:Proc. Mach. Learn. Res. 149, 232–243 (2021)
2021
-
[20]
Johnson, A.E.W., Pollard, T.J., Berkowitz, S.J., Greenbaum, N.R., Lungren, M.P., Deng, C.-Y., Mark, R.G., Horng, S.: MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports.Scientific Data6(1) (2019)
2019
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.