REVIEW 3 major objections 6 minor 1 cited by
The Devil is in the Prompts: De-Identification Traces Enhance Memorization Risks in Synthetic Chest X-Ray Generation
T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read This paper claims that de-identification markers like “___” are the most memorized prompts in text-to-image models trained on MIMIC-CXR, and that the marker token itself contributes most to memorization.
desk verdict A genuinely interesting observation about de-identification markers and high text-conditional norms in MIMIC-CXR, but the paper never verifies that its metric actually detects memorization, and the token-level analysis is underspecified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the memorization score $d_{\mathrm{mem}}$, defined as the average over diffusion timesteps of $\|\epsilon_\theta(x_t, e_p) - \epsilon_\theta(x_t, e_\varnothing)\|_2$: the difference between noise predictions conditioned on the prompt text and on an empty prompt. The paper uses this score to rank all unique prompts, takes the top 1% as memorized, then computes per-token contributions from the same text-conditional norms to isolate the de-identification token “___” as the main driver. The token is distinctive and appears in 21,373 unique prompts, which the paper argues lets the model learn a spurious correlation and use it as a retrieval key.
What would settle it
Generate 50 images from each of the top-1% “___” prompts and compare them against the actual MIMIC-CXR training images with a perceptual similarity metric; if none of the generations is closer to a particular training scan than images from ordinary prompts, the memorization scores are not measuring replication of patient data.
Extended reading notes
Core claim
Using the text-conditional noise metric $d_{\mathrm{mem}}$ (the average $\ell^2$ distance between the noise predictor's outputs when conditioned on a prompt versus on empty text) over all unique MIMIC-CXR prompts, the paper ranks prompts by memorization potential. It finds a heavy-tailed distribution and labels the top one percent as memorized. Token-level attribution for those prompts shows that the Protected Health Information de-identification marker “___” is the single largest contributor to the memorization score, and this pattern repeats across memorized prompts. The paper also reports that replacing the marker with a random word, a random number, or removing it entirely leaves generated images nearly as similar across seeds (mean L2 distance 0.38 before versus 0.42–0.45 after), concluding that inference-time mitigation fails and the issue must be handled at training or curation time.
Load-bearing premise
The whole finding depends on treating the paper's prompt-level memorization score as proof that a prompt makes the model output a near-copy of a specific patient's X-ray, a link the paper does not directly verify.
Editorial extensions
If this is right
- If the marker claim holds, standard anonymization practice for MIMIC-CXR captions actively creates a memorization risk that would not exist with randomized markers.
- Inference-time mitigation—replacing the marker with random words, random numbers, or removing it—leaves the model's outputs nearly unchanged, so the problem must be addressed during training or data curation.
- Other researchers using MIMIC-CXR to train text-to-image models should expect prompts containing “___” to produce seed-invariant, memorized generations.
- The released list of memorized prompts gives a benchmark for testing future memorization mitigation methods for synthetic chest X-ray generation.
- Dataset curators who randomize de-identification markers through a rule-based substitution may reduce caption-level memorization.
Reading between the lines
- If correct, the paper's mechanism generalizes: any uniform, semantically empty placeholder token that appears across many captions could become a retrieval key, so other medical datasets de-identified with a single marker may carry the same latent risk.
- A direct near-duplicate check against MIMIC-CXR originals would strengthen the link between the paper's memorization score and actual patient data exposure, since the paper's evidence is seed-invariance and token attribution rather than image-to-image matching.
- A simple test of causality would be to retrain the model with “___” replaced by randomized per-image tokens and measure whether the top percentile of memorization scores shifts to other tokens.
- Recaptioning or randomizing markers may lower memorization, but could also alter clinically relevant language; quantifying the privacy-utility trade-off is the next open question.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper studies memorization in text-to-image diffusion models trained on MIMIC-CXR chest X-ray data. The authors use the RadEdit model with the detection metric dmem (average L2 distance between text-conditioned and unconditioned noise predictions) to score all unique prompts, label the top 1% as memorized, and then perform a token-level analysis of those prompts. They report that prompts containing de-identification markers such as '___' are the most memorized and that these markers contribute the most to memorization. They further test three inference-time mitigation strategies and conclude that the strategies are ineffective, and they close with recommendations for dataset curators and model developers. The paper also states that a list of memorized prompts and anonymized code will be released.
Significance. If the claims were established, the paper would be a timely contribution: it would be the first systematic prompt- and token-level memorization analysis for medical text-to-image generation, identify an unexpected interaction between standard PHI de-identification markers and retrieval-like memorization, and provide a concrete prompt list for benchmarking. The proposed stakeholder recommendations and the code release add practical value. However, the contribution is currently conditional: the headline findings rest on a memorization detector whose validity is not demonstrated in this setting and on a token-attribution procedure that is not specified in the manuscript. The paper therefore currently provides a plausible hypothesis rather than a verified result.
major comments (3)
- [Sec. 3.2 / Sec. 4.1] The memorization detector dmem is never validated against actual training-image replication. The paper adopts the metric from [27] and treats high dmem as 'potentially memorized' without checking whether high-dmem prompts generate near-duplicates of specific MIMIC-CXR training images. Figure 1 shows only that multiple seeds produce similar images for one prompt; it does not compare those images to the corresponding training radiograph. Seed invariance can also arise from highly specific or unusual prompts even without verbatim copying, and MIMIC-CXR's structured captions and duplicated captions make this confound concrete. If dmem primarily measures prompt specificity or text-embedding distinctiveness, the top-1% selection in Sec. 4.1 and every downstream token-level conclusion would not establish patient-data exposure. I request a validation study: for a sample of high- and low-dmem prompts, compute nearest-neighbor similarity to the training set (e.g., LPIPS or structural similarity to MIMIC-CXR images with the same or nearest caption) and report the replication rate. The authors should also clarify that the prompts considered come from the RadEdit training split; if the full dataset is used, some prompts may be out-of-distribution and memorization would be impossible for them.
- [Sec. 4.2] The token-level attribution method is not described, so the central token-level claim is not reproducible. The paper defines dmem for a full prompt p in Sec. 3.2 but never states how 'the text-conditional norm for each token' in Fig. 3 is obtained. No masking rule, deletion baseline, normalization, or controlled comparison is given. This matters because '___' is a rare, highly distinctive token that frequently co-occurs with long clinical findings; any attribution based on token deletion or on text-conditional norm differences could assign it the largest value by construction. The same dmem concept is used both to select the memorized prompts and to attribute token contributions, so the finding is at risk of circularity. The authors should specify the exact computation, include a sensitivity analysis (e.g., compare against random token removal and against frequency-matched controls), and show the full per-token distribution rather than only the top 25 values.
- [Sec. 4.3] The claim that existing interventions 'fail to sufficiently reduce' memorization is not supported by the reported numbers. The mean inter-seed L2 distance changes from 0.38 to 0.45, 0.43, and 0.42 across the three mitigation strategies, a relative increase of roughly 10-18%, and no standard deviations, confidence intervals, or per-pair distributions are provided; 'nearly unchanged' is therefore an interpretation, not a demonstrated result. In addition, no comparison is made to a control condition with non-de-identification word replacements or to a known non-memorized prompt, so the ineffectiveness claim lacks a baseline. Reporting the full distribution over the 50 samples and a paired statistical test would make the conclusion testable.
minor comments (6)
- [Sec. 4.1] The top-1% threshold for defining memorized prompts is arbitrary; report sensitivity to the threshold (e.g., 0.5%, 2%, 5%) to show that the token-level findings are not artifacts of the cutoff.
- [Fig. 2] The axes are not labeled; the caption says 'largest to smallest' but it is unclear whether the x-axis is rank order and whether the y-axis is dmem. Please add labels and a log-scale option to make the heavy-tailed claim inspectable.
- [Fig. 3] The plot is described as 'the text-conditional norm for each token' but the units and the computation are undefined; at minimum, the caption should state what is being plotted and how the top-25 selection was made.
- [Sec. 1] The statement that 2337 instances share the caption 'No acute cardiopulmonary abnormality.' should state the preprocessing used to obtain the filtered subset and whether punctuation and case normalization were applied, since this affects the uniqueness count.
- [Sec. 3.1] The equation formatting is incomplete: the product expression for αt and the definition of q(xt | xt−1) are typeset incorrectly ('Qt i=1(1 − βt)'). Please fix the math.
- [Abstract / Code] The abstract promises that the anonymized code is 'available here', but the full text does not include the URL; include the link in the published version.
Circularity Check
Central finding reduces to the dmem text-conditional norm used both to define 'memorized prompts' and to score token contributions.
-
self definitional
[Sec. 3.2 (dmem definition), Sec. 4.1 (page 5-6), Sec. 4.2 / Fig. 3 caption]
"dmem = 1/T Σ_t ||ϵθ(xt, ep) − ϵθ(xt, e∅)||2. A higher value of dmem signifies a stronger memorization. ... The prompts corresponding to the top 1 percentile of norm values ... referred to as 'memorized prompts' hereafter ... Figure illustrating the text-conditional norm for each token in a memorized prompt ... the PHI de-identification token ('___') holds the most significant contribution towards memorization."
Memorized prompts are defined by the paper as the top 1% of dmem, the average text-conditional noise norm. The token-level attribution then uses the same text-conditional norm per token and calls it 'contribution towards memorization.' Thus the headline result—that '___' contributes most—is a direct consequence of computing the very norm used to select the prompts, not an independent test of whether that token retrieves actual MIMIC-CXR training images. Because '___' is a rare, distinctive token, its per-token text-conditional norm is expected to be large by the same mechanism that makes prompts containing it score high on dmem, so the 'finding' is built into the metric and the percentile cutoff.
full rationale
The paper's own equations show dmem is the average L2 distance between text-conditioned and unconditioned noise predictions. The paper then defines the top 1% of this score as 'memorized prompts' and, in Sec. 4.2, attributes token-level 'contribution to memorization' using text-conditional norms. Since both the selection and the attribution use the same construct, the claim that de-identification markers are the most memorized tokens is a restatement of the metric rather than an independently verified empirical discovery. The paper does not check that high-dmem prompts actually produce near-duplicates of MIMIC-CXR training images, and the per-token method is unspecified, so the only concrete reduction available is dmem defining what is then 'found' at the token level. There is no load-bearing self-citation chain: the self-citations [6-8] concern PEFT and capacity control and are not used to justify the central finding. The external metric from [27] imports the assumption 'higher dmem = stronger memorization' without grounding in observed replication, which is a correctness risk but not a self-citation circularity. Overall, the central headline reduces by construction to the dmem proxy, so the circularity score is 6.
Assumptions & free parameters
free parameters (3)
- Top 1% dmem percentile cutoff =
top 1% of unique prompts
- Number of generated samples for L2 distance =
50 samples per prompt
- L2 distance change interpreted as 'nearly unchanged' =
0.38 vs 0.45, 0.43, 0.42
assumptions (4)
- domain assumption The dmem metric (text-conditional noise difference) is a reliable memorization detector.
- domain assumption RadEdit contains MIMIC-CXR in its training corpus.
- domain assumption Top 1% of dmem scores correspond to memorized prompts.
- domain assumption Mean L2 distance across random seeds is a valid quantitative measure of memorization, and small changes indicate ineffective mitigation.
Cite this review
Pith. "Pith review of The Devil is in the Prompts: De-Identification Traces Enhance Memorization Risks in Synthetic Chest X-Ray Generation." pith.science (2026). https://pith.science/paper/AQFSF5FC
@misc{pith2026250207516,
author = {Pith},
title = {Pith review of: The Devil is in the Prompts: De-Identification Traces Enhance Memorization Risks in Synthetic Chest X-Ray Generation},
year = {2026},
howpublished = {\url{https://pith.science/paper/AQFSF5FC}},
note = {Machine review of arXiv:2502.07516}
}
read the original abstract
Generative models, particularly text-to-image (T2I) diffusion models, play a crucial role in medical image analysis. However, these models are prone to training data memorization, posing significant risks to patient privacy. Synthetic chest X-ray generation is one of the most common applications in medical image analysis with the MIMIC-CXR dataset serving as the primary data repository for this task. This study presents the first systematic attempt to identify prompts and text tokens in MIMIC-CXR that contribute the most to training data memorization. Our analysis reveals two unexpected findings: (1) prompts containing traces of de-identification procedures (markers introduced to hide Protected Health Information) are the most memorized, and (2) among all tokens, de-identification markers contribute the most towards memorization. This highlights a broader issue with the standard anonymization practices and T2I synthesis with MIMIC-CXR. To exacerbate, existing inference-time memorization mitigation strategies are ineffective and fail to sufficiently reduce the model's reliance on memorized text tokens. On this front, we propose actionable strategies for different stakeholders to enhance privacy and improve the reliability of generative models in medical imaging. Finally, our results provide a foundation for future work on developing and benchmarking memorization mitigation techniques for synthetic chest X-ray generation using the MIMIC-CXR dataset. The anonymized code is available at https://anonymous.4open.science/r/diffusion_memorization-8011/
Figures
Forward citations
Cited by 1 Pith paper
-
Scalable and Private Federated Learning Using Distributed Differential Privacy and Secure Aggregation
DDP-SA combines client-side Laplace noise perturbation with full-threshold additive secret sharing to let federated learning servers reconstruct only aggregated noisy gradients without exposing individual client updates.
Reference graph
Works this paper leans on
-
[27]
In: The Twelfth International Conference on Learning Representations (2024)
Wen, Y., Liu, Y., Chen, C., Lyu, L.: Detecting, explaining, and mitigating memo- rization in diffusion models. In: The Twelfth International Conference on Learning Representations (2024)
work page 2024
-
[1]
Available at SSRN 4611613 (2023)
Akbar, M.U., Wang, W., Eklund, A.: Beware of diffusion models for synthesizing medical images-a comparison with gans in terms of memorizing brain mri and chest x-ray images. Available at SSRN 4611613 (2023)
work page 2023
-
[2]
Bannur, S., Hyland, S., Liu, Q., Pérez-García, F., Ilse, M., Castro, D.C., Boeck- ing, B., Sharma, H., Bouzid, K., Thieme, A., Schwaighofer, A., Wetscherek, M., Lungren, M.P., Nori, A., Alvarez-Valle, J., Oktay, O.: Learning to exploit temporal structure for biomedical vision–language processing. In: Conference on Computer Vision and Pattern Recognition 2...
work page 2023
-
[3]
In: 32nd USENIX Security Symposium (USENIX Security 23)
Carlini, N., Hayes, J., Nasr, M., Jagielski, M., Sehwag, V., Tramer, F., Balle, B., Ippolito, D., Wallace, E.: Extracting training data from diffusion models. In: 32nd USENIX Security Symposium (USENIX Security 23)
-
[4]
Chambon, P., Bluethgen, C., Delbrouck, J.B., Van der Sluijs, R., Połacin, M., Chaves,J.M.Z.,Abraham,T.M.,Purohit,S.,Langlotz,C.P.,Chaudhari,A.:Roent- gen: vision-language foundation model for chest x-ray generation. arXiv:2211.12737
-
[5]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Dar, S.U.H., Ghanaat, A., Kahmann, J., Ayx, I., Papavassiliu, T., Schoenberg, S.O., Engelhardt, S.: Investigating data memorization in 3d latent diffusion mod- els for medical image synthesis. In: International Conference on Medical Image Computing and Computer-Assisted Intervention
-
[6]
In: Med- ical Imaging with Deep Learning (2024), https://openreview.net/forum?id= LVRhXa0q5r
Dutt, R., Ericsson, L., Sanchez, P., Tsaftaris, S.A., Hospedales, T.: Parameter- efficient fine-tuning for medical image analysis: The missed opportunity. In: Med- ical Imaging with Deep Learning (2024), https://openreview.net/forum?id= LVRhXa0q5r
work page 2024
-
[7]
Dutt, R., Sanchez, P., Bohdal, O., Tsaftaris, S.A., Hospedales, T.: Capacity control is an effective memorization mitigation mechanism in text-conditional diffusion models. arXiv preprint arXiv:2410.22149 (2024)
work page Pith review arXiv 2024
Show all 28 references
-
[8]
arXiv preprint arXiv:2405.19458 (2024)
Dutt, R., Sanchez, P., Bohdal, O., Tsaftaris, S.A., Hospedales, T.: Memcontrol: Mitigating memorization in medical diffusion models via automated parameter selection. arXiv preprint arXiv:2405.19458 (2024)
2024 arXiv
-
[9]
arXiv:2306.01322 (2023)
Fernandez, V., Sanchez, P., Pinaya, W.H.L., Jacenków, G., Tsaftaris, S.A., Car- doso, J.: Privacy distillation: reducing re-identification risk of multimodal diffusion models. arXiv:2306.01322 (2023)
2023 arXiv
-
[10]
In: Ghahra- mani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Ghahra- mani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K. (eds.) Ad- vances in Neural Information Processing Systems. vo...
2014
-
[11]
Advances in neural information processing systems (2020)
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in neural information processing systems (2020)
2020
-
[12]
Scientific data
Johnson, A.E., Pollard, T.J., Shen, L., Lehman, L.w.H., Feng, M., Ghassemi, M., Moody,B.,Szolovits,P.,AnthonyCeli,L.,Mark,R.G.:Mimic-iii,afreelyaccessible critical care database. Scientific data
-
[13]
Nature Medicine pp
Ktena, I., Wiles, O., Albuquerque, I., Rebuffi, S.A., Tanno, R., Roy, A.G., Azizi, S., Belgrave, D., Kohli, P., Cemgil, T., et al.: Generative models improve fairness of medical classifiers under distribution shifts. Nature Medicine pp. 1–8 (2024)
2024
-
[14]
sistant for biomedicine in one day
Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., Gao, J.: Llava-med: Training a large language-and-vision as- 10 Raman Dutt et al. sistant for biomedicine in one day. In: Oh, A., Naumann, T., Glober- son, A., Saenko, K., Hardt, M., Levine, S....
2023
-
[15]
Computer Science Review 48, 100546 (2023)
Murtaza, H., Ahmed, M., Khan, N.F., Murtaza, G., Zafar, S., Bano, A.: Synthetic data generation: State of the art in health care domain. Computer Science Review 48, 100546 (2023)
2023
-
[16]
In: European Conference on Computer Vision
Pérez-García, F., Bond-Taylor, S., Sanchez, P.P., van Breugel, B., Castro, D.C., Sharma, H., Salvatelli, V., Wetscherek, M.T., Richardson, H., Lungren, M.P., et al.: Radedit: stress-testing biomedical vision models via diffusion image editing. In: European Conference on Comput...
2024
-
[17]
arXiv preprint arXiv:2307.01952 (2023)
Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rombach, R.: Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 (2023)
2023 arXiv
-
[18]
arXiv:2403.11052 (2024)
Ren, J., Li, Y., Zen, S., Xu, H., Lyu, L., Xing, Y., Tang, J.: Unveiling and mit- igating memorization in text-to-image diffusion models through cross attention. arXiv:2403.11052 (2024)
2024 arXiv
-
[19]
International Journal of Computer Assisted Radiology and Surgery pp
Saragih, D.G., Hibi, A., Tyrrell, P.N.: Using diffusion models to generate synthetic labeled data for medical image segmentation. International Journal of Computer Assisted Radiology and Surgery pp. 1–11 (2024)
2024
-
[20]
Advances in Neural Information Processing Systems
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al.: Laion-5b: An open large- scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems
-
[21]
Segalis, E., Valevski, D., Lumen, D., Matias, Y., Leviathan, Y.: A picture is worth a thousand words: Principled recaptioning improves image generation (2023),https: //arxiv.org/abs/2310.16656
2023 arXiv
-
[22]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Somepalli, G., Singla, V., Goldblum, M., Geiping, J., Goldstein, T.: Diffusion art or digital forgery? investigating data replication in diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6048–6058 (2023)
2023
-
[23]
Advances in Neural Information Processing Systems 36 (2023)
Somepalli, G., Singla, V., Goldblum, M., Geiping, J., Goldstein, T.: Understand- ing and mitigating copying in diffusion models. Advances in Neural Information Processing Systems 36 (2023)
2023
-
[24]
Song,J.,Meng,C.,Ermon,S.:Denoisingdiffusionimplicitmodels.In:International Conference on Learning Representations (2021),https://openreview.net/forum? id=St1giarCHLP
2021
-
[25]
Wang, J., Chung, Y., Ding, Z., Hamm, J.: From majority to minority: A diffusion- based augmentation. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2024 Workshops: ISIC 2024, iMIMIC 2024, EARTH 2024, DeCaF 2024, Held in Conjunction with MICCAI 2024, Marr...
2024
-
[26]
arXiv:2303.12733 (2023)
Webster, R., Rabin, J., Simon, L., Jurie, F.: On the de-duplication of laion-2b. arXiv:2303.12733 (2023)
2023 arXiv
-
[28]
IEEE journal of biomedical and health informatics (2020)
Yoon, J., Drumright, L.N., Van Der Schaar, M.: Anonymization through data syn- thesis using generative adversarial networks (ads-gan). IEEE journal of biomedical and health informatics (2020)
2020
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.