External Validation of Deep Learning Models for BI-RADS Breast Density Prediction from Ultrasound Images

Referee: [Methods / External validation set] External validation set construction (Methods section): The cohort is assembled from 500 future-cancer cases plus 1,500 matched controls; because density is a known risk factor, this case-control design is expected to enrich for higher-density exams relative to a pure screening population. The manuscript must report the actual racial composition percentages (e.g., % Asian, Black, White) for both internal training and external sets, together with a statistical test of difference, to substantiate the claim of “different racial composition.” Without these numbers the generalization interpretation cannot be separated from selection effects.

Authors: We agree that explicit reporting of racial demographics is necessary to support the generalization claim. We will add a dedicated table in the Methods section listing the racial composition percentages (Asian, Black, White, and other) for both the internal training cohort and the external validation set. We will also perform and report a chi-square test (or Fisher's exact test as appropriate) to quantify the statistical difference in distributions. While the case-control sampling does enrich for higher-density breasts, we note that performance is already stratified by density category and that controls were matched on manufacturer and year; the added demographics will allow readers to better isolate racial effects from selection bias. revision: yes

Referee: [Results] Performance reporting (Results): AUROCs are given as point estimates (e.g., micro 0.885, category C 0.70) with no confidence intervals, bootstrap standard errors, or per-model variance across the three architectures. In addition, the risk-model comparison yields p=0.23; the manuscript should state the power calculation or sample-size justification for detecting a clinically meaningful difference in this downstream task.

Authors: We accept the need for improved statistical transparency. We will recompute and report 95% confidence intervals for all AUROCs via bootstrap resampling (1,000 iterations) and will include the standard deviation of AUROC values across the three architectures (DenseNet121, ViT-B/32, ResNet50). For the Tyrer-Cuzick risk-model comparison, we will add a brief sample-size justification based on the 2,000-exam external cohort and note that the study was powered for the primary density-classification task rather than the secondary risk-modeling endpoint. We will also state that the observed p=0.23 indicates the data are consistent with no meaningful difference but acknowledge limited power to detect small AUROC shifts; the point estimates (0.541 vs 0.570) remain informative for the manuscript's conclusions. revision: partial

Referee: [Methods] Independence and harmonization (Methods): The paper asserts the external set is “independent” and drawn from a population with different racial makeup, yet provides no details on site-specific ultrasound acquisition protocols, probe frequencies, or image preprocessing steps that might differ from the training distribution. A table or paragraph comparing key acquisition parameters and confirming no patient overlap is required to support the external-validation framing.

Authors: We will expand the Methods section with a new paragraph describing the external site's ultrasound acquisition protocols, including probe frequencies, transducer types, and gain settings. We will also insert a comparison table listing manufacturer, probe frequency range, image resolution, and preprocessing steps (e.g., resizing, normalization) for both internal and external data. Finally, we will explicitly confirm zero patient overlap by cross-checking unique identifiers and study dates. These additions will provide the necessary technical context to substantiate the external-validation design. revision: yes