REVIEW 4 major objections 5 minor 1 cited by
Interactive Diabetes Risk Prediction Using Explainable Machine Learning: A Dash-Based Approach with SHAP, LIME, and Comorbidity Insights
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper claims that a recall-optimized LightGBM model trained on randomly undersampled 2015 BRFSS data provides the most reliable diabetes risk detection among six tested classifiers, and that embedding it in an explainable web app…
desk verdict A competent but incomplete applied ML prototype: the recall-optimized LightGBM claim is plausible, yet the deployed app presents uncalibrated probabilities as personal risk, which is the load-bearing flaw. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the combination of random undersampling and recall-based model selection: forcing the training set to a 50/50 balance pushes the model to treat diabetic cases as common, raising sensitivity at the expense of precision. LightGBM, a gradient-boosting tree ensemble, carries the prediction, and engineered features, especially the Risk Factor Count, anchor both the model's decisions and its SHAP explanations. SHAP supplies global feature-attribution and local waterfall explanations, LIME gives a second, locally approximated view of individual predictions, and Pearson correlation analysis provides comorbidity flags. The Dash application turns these outputs into a multi-step web form with real-time results, lifestyle and healthcare-access scores, and personalized recommendations.
What would settle it
Re-run the six classifiers on a second, untouched BRFSS year (for example, 2017 or 2019) using the same undersampling strategy, the same recall metric, and the same cross-validation folds; if LightGBM does not again achieve the highest recall with a statistically significant margin, the paper's central model-selection claim is refuted.
Extended reading notes
Core claim
The paper reports that, across three resampling strategies (original, SMOTE, and random undersampling), random undersampling produced the highest recall for detecting diabetic cases, and LightGBM was the model that benefited most consistently from that strategy. ANOVA and Tukey HSD tests on cross-validated recall scores (p-values of $2.13\times10^{-24}$ for SMOTE and $5.39\times10^{-10}$ for undersampling) are presented as evidence that the model differences are statistically significant. The paper therefore selects a recall-optimized LightGBM as the deployed classifier and justifies this choice for health screening by arguing that false negatives are the most costly error. It also reports that an engineered Risk Factor Count, summing high blood pressure, high cholesterol, stroke, and heart disease indicators, is the strongest SHAP contributor to predictions, and that diabetes correlates most strongly with high blood pressure ($r=0.26$) and high cholesterol ($r=0.20$).
Load-bearing premise
The tool's raw probability output comes from a model trained on a 50/50 balanced sample, so the displayed risk percentages assume diabetes is far more common in the user population than it actually is; if that prevalence assumption is wrong, the numbers are systematically inflated even if the ranking of who is at higher risk remains useful.
Editorial extensions
If this is right
- If the recall advantage is real, a screening deployment of this type will catch more true diabetic cases than an accuracy-tuned model, at the cost of more false positives.
- The SHAP and LIME integration lets a user see which factors, such as Risk Factor Count, general health, or BMI, drove their personal risk, enabling targeted lifestyle changes rather than a bare probability.
- Comorbidity flags for hypertension, cholesterol, heart disease, and stroke turn the diabetes prediction into a broader cardiovascular risk prompt.
- The pipeline's modular design suggests it can be adapted to other chronic conditions or to more recent survey data without a full redesign.
- The deployed Dash application demonstrates that explainable machine learning can be delivered to nontechnical users in real time.
Reading between the lines
- The raw probabilities shown to users are likely inflated because the model was trained on a 50/50 sample while real-world type 2 diabetes prevalence is roughly 10–15%; calibrating the output to population prevalence would keep the risk ranking intact while making the displayed percentages honest.
- Because SHAP identifies Risk Factor Count as the dominant predictor, a simple additive risk score may reproduce much of the model's ranking; testing whether a paper-based score matches LightGBM would offer a low-tech screening alternative.
- The same modular pipeline, including undersampling, recall selection, SHAP/LIME, and correlation flags, could be retargeted to other chronic conditions such as hypertension or heart-disease risk by changing the target variable and re-running the comorbidity analysis.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper describes an end-to-end system for diabetes risk prediction using the 2015 CDC BRFSS dataset, with models trained under original, SMOTE, and random-undersampling strategies. LightGBM trained on the undersampled data is selected as the final model based on recall-optimized cross-validation and ANOVA/Tukey HSD tests. The paper also introduces composite lifestyle and healthcare scores, adds SHAP and LIME explanations, and deploys the model as a multi-step Dash web application with personalized recommendations and comorbidity insights.
Significance. If the claims are taken at face value, the paper shows a practical integration of standard ML tools, class-imbalance handling, and explainability for a public-health screening scenario. The use of a large public dataset, comparison across three sampling strategies, statistical significance testing, and an interactive deployment are strengths that make the result relevant to applied ML and health-informatics audiences. However, the central claim that the deployed model provides reliable personal diabetes risk estimates is undermined by the absence of probability calibration after undersampling, by an unexplained SHAP base value, and by the lack of a numeric results table or reproducible artifacts. These issues are fixable, but they are load-bearing for the paper's central contribution.
major comments (4)
- [Sections III.D, IV.A, IV.D] The LightGBM model selected in Section IV.A is trained on data that random undersampling has made approximately class-balanced, while the true BRFSS prevalence is 15.59% diabetic as stated in Section III.B. The raw probabilistic output of such a model encodes the 50/50 training prior, not the population prior. In Section IV.D, the paper reports a '92% probability of diabetes' for a representative individual, but no calibration step (Platt scaling, isotonic regression, or Bayes-factor adjustment) is described anywhere. Because the Dash application presents these probabilities directly to end users as personal diabetes risk, the displayed numbers are systematically inflated unless a calibration adjustment is made. The paper must either add a described and validated calibration procedure or reframe the output as an uncalibrated risk score and clearly state the limitation.
- [Section IV.C, Figure 9] The SHAP waterfall plot in Figure 9 reports a base value of 1.393 in log-odds, which corresponds to a baseline probability of approximately 0.80. This is inconsistent with the claimed 50/50 training prior of the undersampled model and also inconsistent with the population prevalence of 15.59% reported in Section III.B. The paper never explains why the SHAP base value is so high or whether the SHAP explanation was computed on a different model, a different feature encoding, or a transformed target. Without this reconciliation, the interpretability claim in Section IV.C is not internally consistent with the deployment claim.
- [Sections III.E and IV.B] The evaluation protocol is incompletely specified and impedes verification. Section III.E states that GridSearchCV was used for hyperparameter tuning, but it is not stated whether tuning was nested inside the cross-validation folds used to report recall. If hyperparameters were selected on the full dataset before cross-validation, the reported recall values and the ANOVA results in Section IV.B are optimistically biased. Additionally, the paper gives only figures for model performance and no numeric table of recall, precision, F1, or AUC means and standard deviations, and no effect sizes or confidence intervals for the pairwise Tukey HSD comparisons. The central selection claim of 'best recall' cannot be independently assessed from the text as written.
- [Sections V and III.H] The paper presents the Dash application as suitable for real-world public health use, but it provides no external validation on a temporally or geographically distinct dataset and no comparison with established diabetes risk instruments. The conclusion itself lists 'external clinical validation' only as future work (Section V). Given that the deployed system communicates individual risk probabilities to lay users, a held-out validation set, or at minimum a clear explanation of why such validation is unnecessary for the paper's scope, is needed to support the deployment claim.
minor comments (5)
- [Section III.E versus Section IV.A] The list of models is inconsistent: Section III.E names Logistic Regression, Decision Tree, Random Forest, XGBoost, and LightGBM, while Section IV.A states that six classifiers were evaluated, including KNN and Neural Network. Please reconcile the lists.
- [Section IV] Two subsections are labeled IV.D: one covers LIME-Based Local Explanations and the other covers the Interactive Web Interface. The section numbering should be corrected.
- [Section III.C] The 'Lifestyle Score' and 'Healthcare Access Score' are presented to users as personalized scores, but the component weights and the normalization procedure are not defined anywhere. Please provide the exact formulas or, if the weights are arbitrary, state that explicitly.
- [Section III.B] The paper excludes prediabetic entries to form a binary target, but the deployed app is described as a general diabetes risk tool. It should be stated clearly in the interface and limitations that the model cannot distinguish prediabetes from non-diabetes and that prediabetic individuals are treated as non-diabetic in training.
- [Section III.A] The sentence 'No missing values were present in the dataset' is surprising for BRFSS data and should be clarified, since typical BRFSS extracts contain 'Don't know/Refused' categories that must be handled explicitly.
Circularity Check
No circularity found; the paper's claims rest on cross-validated model performance on a public dataset.
full rationale
The paper reports an empirical machine-learning workflow: preprocessing, class-balancing, model comparison via cross-validation, and post-hoc explainability. The central claim (LightGBM with undersampling gives the best recall) is established by measured cross-validation scores on the 2015 BRFSS dataset, not derived from definitions or from fitted parameters renamed as predictions. The composite scores (Lifestyle, Healthcare Access, Risk Factor Count) are hand-defined input features and do not encode the diabetes target; SHAP and LIME explain the fitted model rather than redefine it. No load-bearing self-citation or imported uniqueness theorem appears; all evidence is the authors' own experiments on a public dataset, and model selection is a standard comparison rather than a circular reduction. Therefore no claim can be exhibited as equivalent to its inputs by construction.
Assumptions & free parameters
free parameters (3)
- Lifestyle Score component weights
- Healthcare Access Score component weights
- Undersampling ratio =
balanced 50/50
assumptions (4)
- domain assumption BRFSS 2015 self-reported responses are accurate enough for diabetes risk modeling
- domain assumption Excluding prediabetic cases preserves a meaningful binary prediction task
- standard math SHAP and LIME explanations faithfully represent the model's behavior
- standard math Pearson correlation is sufficient to characterize comorbidity associations
invented entities (2)
-
Composite Lifestyle Score
-
Healthcare Access Score
Cite this review
Pith. "Pith review of Interactive Diabetes Risk Prediction Using Explainable Machine Learning: A Dash-Based Approach with SHAP, LIME, and Comorbidity Insights." pith.science (2026). https://pith.science/paper/TTU3QOHW
@misc{pith2026250505683,
author = {Pith},
title = {Pith review of: Interactive Diabetes Risk Prediction Using Explainable Machine Learning: A Dash-Based Approach with SHAP, LIME, and Comorbidity Insights},
year = {2026},
howpublished = {\url{https://pith.science/paper/TTU3QOHW}},
note = {Machine review of arXiv:2505.05683}
}
read the original abstract
This study presents a web-based interactive health risk prediction tool designed to assess diabetes risk using machine learning models. Built on the 2015 CDC BRFSS dataset, the study evaluates models including Logistic Regression, Random Forest, XGBoost, LightGBM, KNN, and Neural Networks under original, SMOTE, and undersampling strategies. LightGBM with undersampling achieved the best recall, making it ideal for risk detection. The tool integrates SHAP and LIME to explain predictions and highlights comorbidity correlations using Pearson analysis. A Dash-based UI enables user-friendly interaction with model predictions, personalized suggestions, and feature insights, supporting data-driven health awareness.
Forward citations
Cited by 1 Pith paper
-
Trustworthy Chronic Disease Risk Prediction For Self-Directed Preventive Care via Medical Literature Validation
The paper reports that SHAP-identified top features of deep learning models trained on BRFSS 2023 align with medical literature for 13 chronic diseases, and uses this alignment to argue the models are broadly trustworthy.
Reference graph
Works this paper leans on
-
[1]
M.M. Chowdhury, R.S. Ayon, and M.S. Hossain, “Diabetes diagnosis through machine learning: Investigating algorithms and data augmentation for class imbalanced BRFSS dataset,” Preprint, Dept. of Mathematics and Statistics, Texas Tech Univ., Lubbock, TX, USA, and Dept. of Electronics and Telecommunication Eng., Rajshahi Univ. of EngineeringandTechnology,Raj...
work page 2023
-
[2]
Z.Liu,Q.Zhang,H.Zheng,S.Chen,andY.Gong,“Acomparativestudy ofmachinelearningapproachesfordiabetesriskprediction:Insightsfrom SHAP and feature importance,” Preprints, Nov. 19, 2024. [Online]. Available:https://doi.org/10.20944/preprints202411.1265.v1
-
[3]
A comparative analysis of LIME and SHAP interpreters with explainable ML-baseddiabetespredictions,
S. Ahmed, M. S. Kaiser, M. S. Hossain, and K. Andersson, “A comparative analysis of LIME and SHAP interpreters with explainable ML-baseddiabetespredictions,”IEEEAccess,vol.12,pp.87912–87925, 2025.doi:10.1109/ACCESS.2024.3422319
-
[4]
A comparative study of diabetes prediction based on lifestyle factors using machine learning,
B. Nguyen and Y. Zhang, “A comparative study of diabetes prediction based on lifestyle factors using machine learning,” unpublished manuscript,2025
work page 2025
-
[5]
EvaluationofExplainable Artificial Intelligence: SHAP, LIME, and CAM,
H.Nguyen,H.Cao,V.Nguyen,andD.Pham,"EvaluationofExplainable Artificial Intelligence: SHAP, LIME, and CAM," *ResearchGate preprint*, 2021. [Online]. Available: https://www.researchgate.net/publication/362165633
-
[6]
Predictingthe Risk of Diabetes Using Explainable Artificial Intelligence,
A.L.R.Agahan,M.S.A.Magboo,andV.P.C.Magboo,"Predictingthe Risk of Diabetes Using Explainable Artificial Intelligence," in *Proc. 2023 Int. Conf. on Electrical, Computer and Energy Technologies (ICECET)*,2023,pp.1–6.doi:10.1109/ICECET58911.2023.10389419
-
[7]
Explainable AI for healthcare: A study for interpretingdiabetesprediction,
N. Gandhi and S. Mishra, “Explainable AI for healthcare: A study for interpretingdiabetesprediction,”inProc.Int.Conf.onMachineLearning andBigDataAnalytics(ICMLBDA),Cham,Switzerland:Springer,2022, pp.95–105
work page 2022
-
[8]
Y. Ramon, D. Martens, F. Provost, and T. Evgeniou, “A comparison of instance-level counterfactual explanation algorithms for behavioral and textualdata:SEDC,LIME-CandSHAP-C,” Advances inDataAnalysis andClassification,vol.14,no.4,pp.801–819,Dec.2020
work page 2020
Show all 22 references
-
[9]
A diabetes predictionmodelwithvisualizedexplainableartificialintelligence(XAI) technology,
Y. Zhao, J. K. Chaw, M. C. Ang, M. M. Daud, and L. Liu, “A diabetes predictionmodelwithvisualizedexplainableartificialintelligence(XAI) technology,” in *Advances in Visual Informatics (Lecture Notes in ComputerScience)*,vol.14322,H.B.Zaman,Ed.,Singapore:Springer, 2024.doi:10.1...
2024 doi
-
[10]
Correlation based breast cancer detectionusingmachinelearning,
A. Priyadarshini and J. Aravinth, “Correlation based breast cancer detectionusingmachinelearning,”inProc.Int.Conf.onRecentTrendsin Electronics,Information&CommunicationTechnology(RTEICT),2021, pp.499–504
2021
-
[11]
A decision support system for diabetes prediction using machine learning and deep learning techniques,
A. Yahyaoui, A. Jamil, J. Rasheed, and M. Yesiltepe, “A decision support system for diabetes prediction using machine learning and deep learning techniques,” in Proc. 1st Int. Informat. Softw. Eng. Conf. (UBMYK),2019,pp.1–4
2019
-
[12]
Diabetes prediction using machine learningalgorithms,
A. Mujumdar and V. Vaidehi, “Diabetes prediction using machine learningalgorithms,”ProcediaComputerScience,vol.165,pp.292–299, 2019
2019
-
[13]
Machine learning tools for long-term type 2 diabetes risk prediction,
N. Fazakis, O. Kocsis, E. Dritsas, S. Alexiou, N. Fakotakis, and K. Moustakas, “Machine learning tools for long-term type 2 diabetes risk prediction,”IEEEAccess,vol.9,pp.103737–103757,2021
2021
-
[14]
Prediction of diabetes usingmachinelearningalgorithms in healthcare,
M. A. Sarwar, N. Kamal, W. Hamid, and M. A. Shah, “Prediction of diabetes usingmachinelearningalgorithms in healthcare,” in Proc. 24th Int.Conf.onAutomationandComputing(ICAC),Sep.2018,pp.1–6
2018
-
[15]
Primary stage of diabetes prediction using machine learning approaches,
M.U.Emon,M.S.Keya,M.S.Kaiser,M.A.Islam,T.Tanha,andM.S. Zulfiker, “Primary stage of diabetes prediction using machine learning approaches,” in Proc. Int. Conf. on Artificial Intelligence and Smart Systems(ICAIS),Mar.2021,pp.364–367
2021
-
[16]
Doexplanationsreflectdecisions?Amachine-centricstrategyto quantify the performance of explainability algorithms,
Z.Q.Lin,M.J.Shafiee,S.Bochkarev,M.St.Jules,X.Y.Wang,andA. Wong,“Doexplanationsreflectdecisions?Amachine-centricstrategyto quantify the performance of explainability algorithms,” arXiv preprint arXiv:1910.07387,2019
1910 arXiv
-
[17]
The rising burden of non-communicable diseasesinsub-SaharanAfrica,
J. J. Bigna and J. J. Noubiap, “The rising burden of non-communicable diseasesinsub-SaharanAfrica,”TheLancetGlobalHealth,vol.7,no.10, pp.e1295–e1296,2019
2019
-
[18]
Random forests,
L. Breiman, “Random forests,” Machine Learning, vol. 45, pp. 5–32, 2001
2001
-
[19]
Gestationaldiabetesmellitus,
T.A.Buchanan,A.H.Xiang,etal.,“Gestationaldiabetesmellitus,”*The JournalofClinicalInvestigation*,vol.115,no.3,pp.485–491,2005
2005
-
[20]
Managementandpreventionstrategiesfornon-communicable diseases(NCDs)andtheirriskfactors,
A. Budreviciute, S. Damiati, D. K. Sabir, K. Onder, P. Schuller-Goetzburg, G. Plakys, A. Katileviciute, S. Khoja, and R. Kodzius,“Managementandpreventionstrategiesfornon-communicable diseases(NCDs)andtheirriskfactors,”*FrontiersinPublicHealth*,vol. 8,p.788,2020
2020
-
[21]
Handling class imbalance in customer churn prediction,
J. Burez and D. Van den Poel, “Handling class imbalance in customer churn prediction,” Expert Systems with Applications, vol. 36, no. 3, pp. 4626–4636,2009
2009
-
[22]
Aging, diabetes, and the public health system in the United States,
C.J.Caspersen,G.D.Thomas,L.A.Boseman,G.L.Beckles,andA.L. Albright, “Aging, diabetes, and the public health system in the United States,” *American Journal of Public Health*, vol. 102, no. 8, pp. 1482–1497,2012
2012
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.