{"id":"e4f592aa-4f8e-42f7-9983-86b10787814c","arxiv_id":"1908.03272","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A deep learning model classifies duodenal biopsy images into celiac disease, environmental enteropathy, or normal with 92.86% whole-slide accuracy in a small, site-confounded test set.","lead":"Doctors struggle to tell apart two gut diseases that look similar under a microscope. This paper uses a standard image-recognition network to sort biopsy images into celiac disease, environmental enteropathy, or normal tissue, reporting over 90% accuracy on a small test set.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Site confound not removed by stain normalization; EE-vs-rest accuracy may reflect acquisition site rather than disease pathology.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: disease labels are confounded with acquisition site, and the Vahadane normalization is assumed to remove this confound. I agree this is the central threat to the paper's central claim. The proposed site-discrimination test is a direct, feasible check using existing data. Since the reader already assigned CONDITIONAL with moderate confidence, my analysis does not change the verdict; it sharpens the condition. The paper's positive contributions — interpretability analysis via Grad-CAM and layer activations, multi-zoom architecture with explicit negative result, and modest dataset of clinically relevant enteropathies — remain, but the reported numeric accuracy should not be taken at face value until the site confound is tested. An honest non-finding is not appropriate here because the confounding is explicit in the data description and is not internally addressed beyond a presumption of normalization efficacy.","tokens_in":4994,"tokens_out":1911,"duration_ms":21674,"concrete_test":"Train a ResNet50 with the same preprocessing, augmentation, and transfer-learning protocol to classify patch acquisition site (UVA vs Pakistan/Zambia) using the same Vahadane-normalized 1000x1000 patches. If the site classifier achieves high accuracy (e.g., >90%), then site-specific signal survives normalization, and the disease classifier's EE performance is confounded. Conversely, if site classification is near chance, the concern is mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim — 92.86% whole-slide accuracy and macro-AUC 0.99 (Table II) for distinguishing Celiac Disease, Environmental Enteropathy, and Normal — depends on the assumption that the Vahadane stain normalization (Section III-B) eliminates site-specific color and scanner artifacts. This assumption is insecure. In Section II, all EE WSIs are from Aga Khan University (Pakistan) and University Teaching Hospital (Zambia), whereas all CD and Normal WSIs are from the University of Virginia. Thus acquisition site is perfectly confounded with disease class. The near-perfect EE precision and recall (1.00/1.00, Table II) are precisely what a site-discrimination model would produce, since EE occupies a distinct scanner/staining subspace. The paper acknowledges visible color variation across labs and that raw color differences caused erroneous results (Section III-B), but provides no evidence that normalization fully removes this variation. Moreover, no patient-level split is described; if any of the 150 children contribute slides to both train and test, patch-level leakage could inflate accuracy. The reported 0.99 AUC therefore cannot be attributed to disease biology until site and patient-level confounding are addressed.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents a deep learning pipeline to classify duodenal biopsy whole-slide images into Celiac Disease, Environmental Enteropathy, and Normal tissue. Using a ResNet50 pretrained on ImageNet, patch extraction at two magnifications, Vahadane stain normalization, data augmentation, and test-time augmentation, the authors report patch-level accuracies of 88.89% (1000x1000) and 86.82% (2000x2000), and a whole-slide accuracy of 92.86% on 98 held-out slides with macro-AUC 0.99. The paper also presents Grad-CAM and filter visualizations to support model interpretability.","tokens_in":5170,"tokens_out":4677,"duration_ms":44098,"significance":"If the reported results are robust, the approach could offer a reproducible assistive tool for distinguishing histologically overlapping enteropathies, particularly for Environmental Enteropathy, for which computational pathology support is scarce. The paper demonstrates a reasonable transfer-learning setup and includes interpretability analyses. However, the central claim is currently limited by the confounding of disease class with acquisition site, the lack of a patient-level split, and the small EE test set; these limitations prevent the study from establishing that the classifier generalizes on disease biology rather than site-specific artifacts.","major_comments":[{"comment":"The dataset is built so that disease class is completely confounded with acquisition site: all EE WSIs originate from Aga Khan University Hospital and University Teaching Hospital in Zambia, while all CD and Normal WSIs originate from the University of Virginia. Stain normalization to an empirically chosen target image from the EE dataset (Section III-B) is invoked to remove color differences, but no evidence is given that it removes scanner- and protocol-specific batch effects. The fact that unnormalized images yield 'erroneous results' based on color difference indicates the kind of signal the model can exploit; without a quantitative demonstration that normalization eliminates site-discriminative information (e.g., a site-classification control experiment), the perfect EE precision/recall in Table II can be explained by acquisition site. This must be addressed with a site-holdout evaluation or an explicit normalization-validation experiment.","section":"Section II and Section III-B"},{"comment":"The paper does not describe whether the 367 training and 98 test slides are disjoint at the patient level. With 150 children contributing 465 WSIs, multiple WSIs per child are present; if slides from the same patient appear in both sets, patch-level leakage can inflate the reported accuracy. The authors should confirm that the split is by patient and, if it is not, repeat the evaluation with patient-level cross-validation.","section":"Section II / IV-A"},{"comment":"The EE test set contains only 11 whole-slide images, and the paper reports no confidence intervals or statistical tests for the 92.86% accuracy or macro-AUC of 0.99. With n=11, the 1.00 precision/recall for EE is not a stable estimate; a single error changes the metric by roughly 9 percentage points. The abstract's 'over 90%' claim should be qualified accordingly.","section":"Table II / Section IV-A"}],"minor_comments":[{"comment":"The phrase 'unlabeled 98 slides' is misleading because Table II provides the true class support counts; rename this to 'held-out test slides'.","section":"Section IV-A"},{"comment":"The text states 'Figure 3 displays a few example Grad-CAM outputs', but Figure 3 is the ROC curves; the Grad-CAM images appear to be in Figure 2. Please correct the cross-references.","section":"Section IV-B"},{"comment":"Test-time augmentation is described as random during prediction; for reproducible and stable predictions, use a fixed set of augmentations and average the outputs.","section":"Section III-C"},{"comment":"The multi-zoom approach is described but no quantitative results are reported; either present the comparison or remove the section.","section":"Section III-E"},{"comment":"The relationship between patients, WSIs, and slides ('465 high resolution whole slide images taken from 150 H&E duodenal biopsy slides') is unclear; clarify the unit of analysis and state whether multiple WSIs from the same patient are treated as independent.","section":"Section II"},{"comment":"Reference [6] is a prior abstract by the same group; the paper should explicitly state the new contribution beyond that abstract.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"This is a clearly written paper addressing a real clinical problem, but the site confound is fundamental and currently undermines the central claim. The authors should be given the chance to address it with additional experiments, such as a site-holdout validation or a demonstration that stain normalization removes site-discriminative features. The paper may be more appropriate for a clinical informatics venue after such revisions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a plausible transfer-learning result for a real clinical problem, but the headline accuracy rests on a site confound the paper never addresses. The new bit is the three-way CD/EE/Normal classification on multi-site data, plus the interpretability analysis. The methods are standard ResNet50 with fine-tuning, stain normalization, and test-time augmentation; nothing fancy, but the engineering is reasonable and the paper is honest that the multi-scale fusion didn't help.\n\nWhat's good: the dataset is genuinely multi-site (Pakistan, Zambia, US), which is rare; they report whole-slide level accuracy after patch aggregation; they show Grad-CAM maps and filter visualizations, which is useful for clinicians; and they acknowledge the color variation problem and attempt Vahadane normalization. The EE class being perfectly separated (precision/recall 1.00 on 11 slides) is either a great result or a red flag.\n\nThe red flag: acquisition site is perfectly confounded with disease. All EE slides come from Pakistan/Zambia; all CD and Normal from UVA. Stain normalization is supposed to remove scanner/staining differences, but there is no validation that it does. The paper itself says raw color differences led to erroneous results; normalization is empirical (target chosen from EE), and no analysis shows post-normalization color distributions overlap across sites. So the classifier could be learning site identity. That would explain the perfect EE scores. Also, no patient-level split is described; if the 150 children overlap train/test, patch leakage inflates accuracy. The test set is only 98 slides with 11 EE, so the macro-AUC 0.99 has wide error bars; no CIs are given.\n\nThe abstract's 'over 90%' claim refers to the whole-slide accuracy, which is fair, but the patch-level numbers are lower. Minor inconsistencies in slide counts (text vs Table I) suggest the data pipeline is not fully documented. The paper cites its own prior abstract [6]; that's fine as prior work, not a problem.\n\nBottom line: the paper is a legitimate exploratory study for a clinically meaningful differential diagnosis. It should go to peer review, but the revision needs to address the site confound and patient-level splitting. If the authors can show the result holds within-site (e.g., UVA-only EE vs others, or external validation), the claim will be credible. As it stands, the 92.86% is not yet attributable to disease biology.\n\nI'd bring it to reading group as a cautionary example, but I wouldn't cite it yet.","headline":"Plausible transfer-learning result for a hard differential diagnosis, but the perfect EE scores smell like site confound; needs a within-site or patient-level check before the accuracy is believable.","tokens_in":5777,"tokens_out":2373,"would_cite":false,"duration_ms":23527,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A ResNet50 trained on color-normalized patches of H&E duodenal biopsies can separate Celiac Disease, Environmental Enteropathy, and Normal tissue, reporting 92.86% whole-slide accuracy and macro-average AUC 0.99.","keywords":["deep learning","duodenal biopsy","celiac disease","environmental enteropathy","whole slide images","transfer learning","stain normalization","Grad-CAM"],"falsifier":"Take the same three-class task but train and test on slides drawn from every site, or from one site scanned on multiple scanners, and see whether accuracy stays above 90%. A second check: train a classifier on the stain-normalized patches' color histograms alone; if it separates classes nearly as well, the visual features are dominated by residual site artifacts.","tokens_in":4760,"feed_emoji":"🔬","tokens_out":6320,"duration_ms":66708,"temperature":0.7,"pith_summary":"The paper tries to establish that a deep convolutional network can separate three histologically similar duodenal biopsy categories—Celiac Disease, Environmental Enteropathy, and Normal—at an accuracy high enough to be a plausible assistive diagnostic. It reports whole-slide accuracy of 92.86%, with perfect precision and recall for Environmental Enteropathy and a macro-average AUC of 0.99. This matters because EE and Celiac Disease share overlapping features that can confuse human pathologists, so a reproducible computational read could help standardize diagnosis. The paper also argues that the model's attention maps focus on pathology-like structures rather than obvious image artifacts.","feed_headline":"Deep learning tells celiac disease from environmental enteropathy","feed_subtitle":"ResNet on duodenal biopsy patches hits 92.86% whole-slide accuracy and 0.99 macro AUC.","key_machinery":"The load-bearing object is a ResNet50 convolutional network pre-trained on a large natural-image dataset, fine-tuned discriminatively with cyclic cosine annealing and test-time augmentation, and trained on 256-by-256 resized patches extracted from whole-slide images. A structure-preserving stain-normalization step maps all slides to a common color appearance, intended to strip scanner and laboratory variation so the network learns tissue features instead of color differences. Patch-level predictions are majority-aggregated to whole-slide labels; Grad-CAM and filter visualizations are then used to inspect which tissue regions and intermediate features drive the classification.","core_discovery":"On the paper's own terms, the central claim is that transfer-learned ResNet50 models applied to color-normalized patches of H&E duodenal biopsies classify the three classes at the whole-slide level with 92.86% accuracy and a macro-average AUC of 0.99. Patch-level predictions are aggregated to whole-slide labels; Environmental Enteropathy is recognized with perfect precision and recall, while Celiac Disease and Normal show more confusion with each other. The paper reports that fusing features from two magnification levels gave little benefit over single-scale models, and that visual explanation methods suggest the network attends to nuclei, cell boundaries, and tissue-background patterns rather than pure color differences.","pith_inferences":["My inference: because all Environmental Enteropathy slides came from two clinical sites while Celiac Disease and Normal slides came from a third site, the 0.99 AUC may partly encode site or scanner identity rather than disease biology; the paper does not test this directly.","A cross-site validation set with EE, Celiac Disease, and Normal slides drawn from every site would settle whether stain normalization fully removes the site confound; until then, site identity remains a plausible alternative explanation for EE's perfect recall.","The choice of the stain-normalization target image drawn from the EE set is a free parameter; my expectation is that changing the target could shift the decision boundary, and sensitivity to that choice should be reported.","If the method transfers to new sites, an obvious next step is weakly supervised whole-slide classification that skips explicit patch aggregation, which would remove the need for the multi-zoom fusion the paper found unhelpful."],"forward_implications":["If the reported accuracy holds in clinical settings, automated triage of duodenal biopsies could flag likely Celiac Disease or Environmental Enteropathy cases before expert review.","The near-perfect recognition of Environmental Enteropathy suggests screening for EE in populations with growth faltering could be automated from routine biopsy slides.","The finding that multi-zoom feature fusion adds little performance benefit implies single-magnification patch models may be sufficient for similar histopathology classification tasks.","The interpretability evidence indicates the model's decisions can be aligned with the features pathologists look for, supporting use as a second-reader tool.","The same transfer-learning pipeline could plausibly be applied to other enteropathies with overlapping histology."],"supporting_citations":[{"why":"Establishes the precedent of applying convolutional neural networks to whole-slide histopathology images for lesion detection, which this paper extends to enteropathies.","marker":"[7]"},{"why":"Reports an earlier CNN-based automated diagnosis of celiac disease and supplies a direct benchmark for the task.","marker":"[8]"},{"why":"Describes a deep learning approach for celiac disease detection on duodenal biopsy slides, the closest prior work on this tissue type.","marker":"[9]"},{"why":"Supplies the stain-normalization step that is intended to remove color and scanner variation before classification.","marker":"[12]"},{"why":"Justifies the ResNet50 architecture because it performed well in computer-assisted breast cancer diagnosis.","marker":"[13]"},{"why":"Motivates transfer learning from a large pretrained model as a response to the small biopsy dataset.","marker":"[14]"},{"why":"Provides the large-scale natural-image dataset used for the pretrained weights that initialize the model.","marker":"[15]"},{"why":"Describes discriminative fine-tuning, the layer-dependent learning-rate schedule used during training.","marker":"[16]"},{"why":"Supplies the Grad-CAM localization method used to show which tissue regions drive the classification decisions.","marker":"[18]"},{"why":"Supplies the filter-visualization technique used to check that intermediate layers detect nuclei, cell boundaries, and background.","marker":"[19]"}],"fun_headline_variants":["Deep learning distinguishes celiac and environmental enteropathy at 92.86% accuracy","Deep learning separates celiac from environmental enteropathy in biopsies","Neural net reads duodenal biopsies, tells celiac from environmental enteropathy","Deep learning decodes celiac vs environmental enteropathy on biopsy slides","AI separates two enteropathies in duodenal biopsies"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire result depends on the assumption that the stain-normalization step erases the color and scanner differences between the sites where the biopsies were collected, so that the network is learning disease biology rather than which hospital produced the slide.","fun_headline_variants_meta":{"raw":{"variants":["Deep learning distinguishes celiac and environmental enteropathy at 92.86% accuracy","Deep learning separates celiac from environmental enteropathy in biopsies","Neural net reads duodenal biopsies, tells celiac from environmental enteropathy","Deep learning decodes celiac vs environmental enteropathy on biopsy slides","AI separates two enteropathies in duodenal biopsies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001471,"raw_usage":{"total_tokens":5840,"prompt_tokens":798,"completion_tokens":5042,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":414,"completion_tokens_details":{"reasoning_tokens":4951}},"tokens_in":414,"tokens_out":5042,"duration_ms":38745,"temperature":1.0,"reasoning_tokens":4951,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:18:38.644555+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same three-class task but train and test on slides drawn from every site, or from one site scanned on multiple scanners, and see whether accuracy stays above 90%. A second check: train a classifier on the stain-normalized patches' color histograms alone; if it separates classes nearly as well, the visual features are dominated by residual site artifacts.","supporting_citations":[{"cited_title":"Cruz-Roa, A","cited_arxiv_id":null,"evidence_quote":"Establishes the precedent of applying convolutional neural networks to whole-slide histopathology images for lesion detection, which this paper extends to enteropathies."},{"cited_title":"”Convolutional neural network architectures for the automated diagnosis of celiac disease.” International Workshop on Computer-Assisted and Robotic Endoscopy","cited_arxiv_id":null,"evidence_quote":"Reports an earlier CNN-based automated diagnosis of celiac disease and supplies a direct benchmark for the task."},{"cited_title":"Automated detection of celiac disease on duodenal biopsy slides: a deep learning approach","cited_arxiv_id":"1901.11447","evidence_quote":"Describes a deep learning approach for celiac disease detection on duodenal biopsy slides, the closest prior work on this tissue type."},{"cited_title":"Vahadane, T","cited_arxiv_id":null,"evidence_quote":"Supplies the stain-normalization step that is intended to remove color and scanner variation before classification."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Justifies the ResNet50 architecture because it performed well in computer-assisted breast cancer diagnosis."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Motivates transfer learning from a large pretrained model as a response to the small biopsy dataset."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the large-scale natural-image dataset used for the pretrained weights that initialize the model."},{"cited_title":"Howard and S","cited_arxiv_id":null,"evidence_quote":"Describes discriminative fine-tuning, the layer-dependent learning-rate schedule used during training."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Grad-CAM localization method used to show which tissue regions drive the classification decisions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the filter-visualization technique used to check that intermediate layers detect nuclei, cell boundaries, and background."}],"review_version":1}