Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Interactive Diabetes Risk Prediction Using Explainable Machine Learning: A Dash-Based Approach with SHAP, LIME, and Comorbidity Insights

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Pith's one-line read This paper claims that a recall-optimized LightGBM model trained on randomly undersampled 2015 BRFSS data provides the most reliable diabetes risk detection among six tested classifiers, and that embedding it in an explainable web app…

desk verdict A competent but incomplete applied ML prototype: the recall-optimized LightGBM claim is plausible, yet the deployed app presents uncalibrated probabilities as personal risk, which is the load-bearing flaw. read the letter →

arxiv 2505.05683 v1 pith:TTU3QOHW submitted 2025-05-08 cs.LG cs.AI

classification cs.LGcs.AI
keywords diabetesriskpredictionBRFSSLightGBMrandomundersamplingclassimbalanceSHAPLIMEDashapplication
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a complete, explainable diabetes-risk screening pipeline can be built from a large self-reported public-health survey, and that the choice of class-balancing technique matters as much as the choice of classifier. It reports that LightGBM trained on randomly undersampled BRFSS data, with diabetic and non-diabetic cases forced into a roughly 50/50 split, achieves the best recall among six classifiers, and argues that maximizing recall is the right objective for screening because missed diabetes cases delay care. To make the model usable, the paper wraps it in a Dash web application that shows SHAP and LIME explanations for each prediction, computes composite lifestyle and healthcare-access scores, and flags correlated comorbidities such as high blood pressure and high cholesterol. A sympathetic reader would care because the paper offers a template for turning a black-box classifier into a public-facing risk communication tool.

What carries the argument

The load-bearing mechanism is the combination of random undersampling and recall-based model selection: forcing the training set to a 50/50 balance pushes the model to treat diabetic cases as common, raising sensitivity at the expense of precision. LightGBM, a gradient-boosting tree ensemble, carries the prediction, and engineered features, especially the Risk Factor Count, anchor both the model's decisions and its SHAP explanations. SHAP supplies global feature-attribution and local waterfall explanations, LIME gives a second, locally approximated view of individual predictions, and Pearson correlation analysis provides comorbidity flags. The Dash application turns these outputs into a multi-step web form with real-time results, lifestyle and healthcare-access scores, and personalized recommendations.

What would settle it

Re-run the six classifiers on a second, untouched BRFSS year (for example, 2017 or 2019) using the same undersampling strategy, the same recall metric, and the same cross-validation folds; if LightGBM does not again achieve the highest recall with a statistically significant margin, the paper's central model-selection claim is refuted.

Watch

Extended reading notes

Core claim

The paper reports that, across three resampling strategies (original, SMOTE, and random undersampling), random undersampling produced the highest recall for detecting diabetic cases, and LightGBM was the model that benefited most consistently from that strategy. ANOVA and Tukey HSD tests on cross-validated recall scores (p-values of $2.13\times10^{-24}$ for SMOTE and $5.39\times10^{-10}$ for undersampling) are presented as evidence that the model differences are statistically significant. The paper therefore selects a recall-optimized LightGBM as the deployed classifier and justifies this choice for health screening by arguing that false negatives are the most costly error. It also reports that an engineered Risk Factor Count, summing high blood pressure, high cholesterol, stroke, and heart disease indicators, is the strongest SHAP contributor to predictions, and that diabetes correlates most strongly with high blood pressure ($r=0.26$) and high cholesterol ($r=0.20$).

Load-bearing premise

The tool's raw probability output comes from a model trained on a 50/50 balanced sample, so the displayed risk percentages assume diabetes is far more common in the user population than it actually is; if that prevalence assumption is wrong, the numbers are systematically inflated even if the ranking of who is at higher risk remains useful.

Editorial extensions

If this is right

  • If the recall advantage is real, a screening deployment of this type will catch more true diabetic cases than an accuracy-tuned model, at the cost of more false positives.
  • The SHAP and LIME integration lets a user see which factors, such as Risk Factor Count, general health, or BMI, drove their personal risk, enabling targeted lifestyle changes rather than a bare probability.
  • Comorbidity flags for hypertension, cholesterol, heart disease, and stroke turn the diabetes prediction into a broader cardiovascular risk prompt.
  • The pipeline's modular design suggests it can be adapted to other chronic conditions or to more recent survey data without a full redesign.
  • The deployed Dash application demonstrates that explainable machine learning can be delivered to nontechnical users in real time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The raw probabilities shown to users are likely inflated because the model was trained on a 50/50 sample while real-world type 2 diabetes prevalence is roughly 10–15%; calibrating the output to population prevalence would keep the risk ranking intact while making the displayed percentages honest.
  • Because SHAP identifies Risk Factor Count as the dominant predictor, a simple additive risk score may reproduce much of the model's ranking; testing whether a paper-based score matches LightGBM would offer a low-tech screening alternative.
  • The same modular pipeline, including undersampling, recall selection, SHAP/LIME, and correlation flags, could be retargeted to other chronic conditions such as hypertension or heart-disease risk by changing the target variable and re-running the comorbidity analysis.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper describes an end-to-end system for diabetes risk prediction using the 2015 CDC BRFSS dataset, with models trained under original, SMOTE, and random-undersampling strategies. LightGBM trained on the undersampled data is selected as the final model based on recall-optimized cross-validation and ANOVA/Tukey HSD tests. The paper also introduces composite lifestyle and healthcare scores, adds SHAP and LIME explanations, and deploys the model as a multi-step Dash web application with personalized recommendations and comorbidity insights.

Significance. If the claims are taken at face value, the paper shows a practical integration of standard ML tools, class-imbalance handling, and explainability for a public-health screening scenario. The use of a large public dataset, comparison across three sampling strategies, statistical significance testing, and an interactive deployment are strengths that make the result relevant to applied ML and health-informatics audiences. However, the central claim that the deployed model provides reliable personal diabetes risk estimates is undermined by the absence of probability calibration after undersampling, by an unexplained SHAP base value, and by the lack of a numeric results table or reproducible artifacts. These issues are fixable, but they are load-bearing for the paper's central contribution.

major comments (4)
  1. [Sections III.D, IV.A, IV.D] The LightGBM model selected in Section IV.A is trained on data that random undersampling has made approximately class-balanced, while the true BRFSS prevalence is 15.59% diabetic as stated in Section III.B. The raw probabilistic output of such a model encodes the 50/50 training prior, not the population prior. In Section IV.D, the paper reports a '92% probability of diabetes' for a representative individual, but no calibration step (Platt scaling, isotonic regression, or Bayes-factor adjustment) is described anywhere. Because the Dash application presents these probabilities directly to end users as personal diabetes risk, the displayed numbers are systematically inflated unless a calibration adjustment is made. The paper must either add a described and validated calibration procedure or reframe the output as an uncalibrated risk score and clearly state the limitation.
  2. [Section IV.C, Figure 9] The SHAP waterfall plot in Figure 9 reports a base value of 1.393 in log-odds, which corresponds to a baseline probability of approximately 0.80. This is inconsistent with the claimed 50/50 training prior of the undersampled model and also inconsistent with the population prevalence of 15.59% reported in Section III.B. The paper never explains why the SHAP base value is so high or whether the SHAP explanation was computed on a different model, a different feature encoding, or a transformed target. Without this reconciliation, the interpretability claim in Section IV.C is not internally consistent with the deployment claim.
  3. [Sections III.E and IV.B] The evaluation protocol is incompletely specified and impedes verification. Section III.E states that GridSearchCV was used for hyperparameter tuning, but it is not stated whether tuning was nested inside the cross-validation folds used to report recall. If hyperparameters were selected on the full dataset before cross-validation, the reported recall values and the ANOVA results in Section IV.B are optimistically biased. Additionally, the paper gives only figures for model performance and no numeric table of recall, precision, F1, or AUC means and standard deviations, and no effect sizes or confidence intervals for the pairwise Tukey HSD comparisons. The central selection claim of 'best recall' cannot be independently assessed from the text as written.
  4. [Sections V and III.H] The paper presents the Dash application as suitable for real-world public health use, but it provides no external validation on a temporally or geographically distinct dataset and no comparison with established diabetes risk instruments. The conclusion itself lists 'external clinical validation' only as future work (Section V). Given that the deployed system communicates individual risk probabilities to lay users, a held-out validation set, or at minimum a clear explanation of why such validation is unnecessary for the paper's scope, is needed to support the deployment claim.
minor comments (5)
  1. [Section III.E versus Section IV.A] The list of models is inconsistent: Section III.E names Logistic Regression, Decision Tree, Random Forest, XGBoost, and LightGBM, while Section IV.A states that six classifiers were evaluated, including KNN and Neural Network. Please reconcile the lists.
  2. [Section IV] Two subsections are labeled IV.D: one covers LIME-Based Local Explanations and the other covers the Interactive Web Interface. The section numbering should be corrected.
  3. [Section III.C] The 'Lifestyle Score' and 'Healthcare Access Score' are presented to users as personalized scores, but the component weights and the normalization procedure are not defined anywhere. Please provide the exact formulas or, if the weights are arbitrary, state that explicitly.
  4. [Section III.B] The paper excludes prediabetic entries to form a binary target, but the deployed app is described as a general diabetes risk tool. It should be stated clearly in the interface and limitations that the model cannot distinguish prediabetes from non-diabetes and that prediabetic individuals are treated as non-diabetic in training.
  5. [Section III.A] The sentence 'No missing values were present in the dataset' is surprising for BRFSS data and should be clarified, since typical BRFSS extracts contain 'Don't know/Refused' categories that must be handled explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; the paper's claims rest on cross-validated model performance on a public dataset.

full rationale

The paper reports an empirical machine-learning workflow: preprocessing, class-balancing, model comparison via cross-validation, and post-hoc explainability. The central claim (LightGBM with undersampling gives the best recall) is established by measured cross-validation scores on the 2015 BRFSS dataset, not derived from definitions or from fitted parameters renamed as predictions. The composite scores (Lifestyle, Healthcare Access, Risk Factor Count) are hand-defined input features and do not encode the diabetes target; SHAP and LIME explain the fitted model rather than redefine it. No load-bearing self-citation or imported uniqueness theorem appears; all evidence is the authors' own experiments on a public dataset, and model selection is a standard comparison rather than a circular reduction. Therefore no claim can be exhibited as equivalent to its inputs by construction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 2 invented entities

The central model is an empirical classifier on a public dataset, so the main ledger entries are hand-defined composite scores and the undersampling choice. The prediction probabilities shown to users inherit the 50/50 class prior, which is a free modeling choice that affects interpretation.

free parameters (3)
  • Lifestyle Score component weights
    The score aggregates physical activity, diet, smoking, alcohol, and poor-health days but no exact weights or formula are given (Section III.C).
  • Healthcare Access Score component weights
    The score aggregates insurance, affordability, and access responses; exact construction is not specified (Section III.C).
  • Undersampling ratio = balanced 50/50
    Random undersampling reduced the majority class to match the minority class (Section III.D), a hand-chosen strategy that alters the class prior.
assumptions (4)
  • domain assumption BRFSS 2015 self-reported responses are accurate enough for diabetes risk modeling
    The entire model relies on self-reported survey data without verification (Section III.A).
  • domain assumption Excluding prediabetic cases preserves a meaningful binary prediction task
    The paper drops prediabetic entries and recodes the target to binary without analyzing the excluded population (Section III.B).
  • standard math SHAP and LIME explanations faithfully represent the model's behavior
    The paper applies these tools as trustworthy interpretability methods (Section III.F).
  • standard math Pearson correlation is sufficient to characterize comorbidity associations
    The comorbidity insights rely on simple linear correlation without controlling for confounders (Section III.G).
invented entities (2)
  • Composite Lifestyle Score
    purpose: Summarizes behavioral risk factors into a single 1 to 5 score shown in the app
    No external validation or formal definition of constituent weights is provided (Section III.C).
  • Healthcare Access Score
    purpose: Summarizes insurance and access-to-care responses for the user interface
    Same lack of validation and specification (Section III.C).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interactive Diabetes Risk Prediction Using Explainable Machine Learning: A Dash-Based Approach with SHAP, LIME, and Comorbidity Insights." pith.science (2026). https://pith.science/paper/TTU3QOHW

@misc{pith2026250505683,
  author       = {Pith},
  title        = {Pith review of: Interactive Diabetes Risk Prediction Using Explainable Machine Learning: A Dash-Based Approach with SHAP, LIME, and Comorbidity Insights},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TTU3QOHW}},
  note         = {Machine review of arXiv:2505.05683}
}
read the original abstract

This study presents a web-based interactive health risk prediction tool designed to assess diabetes risk using machine learning models. Built on the 2015 CDC BRFSS dataset, the study evaluates models including Logistic Regression, Random Forest, XGBoost, LightGBM, KNN, and Neural Networks under original, SMOTE, and undersampling strategies. LightGBM with undersampling achieved the best recall, making it ideal for risk detection. The tool integrates SHAP and LIME to explain predictions and highlights comorbidity correlations using Pearson analysis. A Dash-based UI enables user-friendly interaction with model predictions, personalized suggestions, and feature insights, supporting data-driven health awareness.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Trustworthy Chronic Disease Risk Prediction For Self-Directed Preventive Care via Medical Literature Validation

    cs.LG 2025-06 reject novelty 4.0 of 10

    The paper reports that SHAP-identified top features of deep learning models trained on BRFSS 2023 align with medical literature for 13 chronic diseases, and uses this alignment to argue the models are broadly trustworthy.

Reference graph

Works this paper leans on

22 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [1]

    Diabetes diagnosis through machine learning: Investigating algorithms and data augmentation for class imbalanced BRFSS dataset,

    M.M. Chowdhury, R.S. Ayon, and M.S. Hossain, “Diabetes diagnosis through machine learning: Investigating algorithms and data augmentation for class imbalanced BRFSS dataset,” Preprint, Dept. of Mathematics and Statistics, Texas Tech Univ., Lubbock, TX, USA, and Dept. of Electronics and Telecommunication Eng., Rajshahi Univ. of EngineeringandTechnology,Raj...

  2. [2]

    Acomparativestudy ofmachinelearningapproachesfordiabetesriskprediction:Insightsfrom SHAP and feature importance,

    Z.Liu,Q.Zhang,H.Zheng,S.Chen,andY.Gong,“Acomparativestudy ofmachinelearningapproachesfordiabetesriskprediction:Insightsfrom SHAP and feature importance,” Preprints, Nov. 19, 2024. [Online]. Available:https://doi.org/10.20944/preprints202411.1265.v1

  3. [3]

    A comparative analysis of LIME and SHAP interpreters with explainable ML-baseddiabetespredictions,

    S. Ahmed, M. S. Kaiser, M. S. Hossain, and K. Andersson, “A comparative analysis of LIME and SHAP interpreters with explainable ML-baseddiabetespredictions,”IEEEAccess,vol.12,pp.87912–87925, 2025.doi:10.1109/ACCESS.2024.3422319

  4. [4]

    A comparative study of diabetes prediction based on lifestyle factors using machine learning,

    B. Nguyen and Y. Zhang, “A comparative study of diabetes prediction based on lifestyle factors using machine learning,” unpublished manuscript,2025

  5. [5]

    EvaluationofExplainable Artificial Intelligence: SHAP, LIME, and CAM,

    H.Nguyen,H.Cao,V.Nguyen,andD.Pham,"EvaluationofExplainable Artificial Intelligence: SHAP, LIME, and CAM," *ResearchGate preprint*, 2021. [Online]. Available: https://www.researchgate.net/publication/362165633

  6. [6]

    Predictingthe Risk of Diabetes Using Explainable Artificial Intelligence,

    A.L.R.Agahan,M.S.A.Magboo,andV.P.C.Magboo,"Predictingthe Risk of Diabetes Using Explainable Artificial Intelligence," in *Proc. 2023 Int. Conf. on Electrical, Computer and Energy Technologies (ICECET)*,2023,pp.1–6.doi:10.1109/ICECET58911.2023.10389419

  7. [7]

    Explainable AI for healthcare: A study for interpretingdiabetesprediction,

    N. Gandhi and S. Mishra, “Explainable AI for healthcare: A study for interpretingdiabetesprediction,”inProc.Int.Conf.onMachineLearning andBigDataAnalytics(ICMLBDA),Cham,Switzerland:Springer,2022, pp.95–105

  8. [8]

    A comparison of instance-level counterfactual explanation algorithms for behavioral and textualdata:SEDC,LIME-CandSHAP-C,

    Y. Ramon, D. Martens, F. Provost, and T. Evgeniou, “A comparison of instance-level counterfactual explanation algorithms for behavioral and textualdata:SEDC,LIME-CandSHAP-C,” Advances inDataAnalysis andClassification,vol.14,no.4,pp.801–819,Dec.2020

Show all 22 references
  1. [9]

    A diabetes predictionmodelwithvisualizedexplainableartificialintelligence(XAI) technology,

    Y. Zhao, J. K. Chaw, M. C. Ang, M. M. Daud, and L. Liu, “A diabetes predictionmodelwithvisualizedexplainableartificialintelligence(XAI) technology,” in *Advances in Visual Informatics (Lecture Notes in ComputerScience)*,vol.14322,H.B.Zaman,Ed.,Singapore:Springer, 2024.doi:10.1...

  2. [10]

    Correlation based breast cancer detectionusingmachinelearning,

    A. Priyadarshini and J. Aravinth, “Correlation based breast cancer detectionusingmachinelearning,”inProc.Int.Conf.onRecentTrendsin Electronics,Information&CommunicationTechnology(RTEICT),2021, pp.499–504

  3. [11]

    A decision support system for diabetes prediction using machine learning and deep learning techniques,

    A. Yahyaoui, A. Jamil, J. Rasheed, and M. Yesiltepe, “A decision support system for diabetes prediction using machine learning and deep learning techniques,” in Proc. 1st Int. Informat. Softw. Eng. Conf. (UBMYK),2019,pp.1–4

  4. [12]

    Diabetes prediction using machine learningalgorithms,

    A. Mujumdar and V. Vaidehi, “Diabetes prediction using machine learningalgorithms,”ProcediaComputerScience,vol.165,pp.292–299, 2019

  5. [13]

    Machine learning tools for long-term type 2 diabetes risk prediction,

    N. Fazakis, O. Kocsis, E. Dritsas, S. Alexiou, N. Fakotakis, and K. Moustakas, “Machine learning tools for long-term type 2 diabetes risk prediction,”IEEEAccess,vol.9,pp.103737–103757,2021

  6. [14]

    Prediction of diabetes usingmachinelearningalgorithms in healthcare,

    M. A. Sarwar, N. Kamal, W. Hamid, and M. A. Shah, “Prediction of diabetes usingmachinelearningalgorithms in healthcare,” in Proc. 24th Int.Conf.onAutomationandComputing(ICAC),Sep.2018,pp.1–6

  7. [15]

    Primary stage of diabetes prediction using machine learning approaches,

    M.U.Emon,M.S.Keya,M.S.Kaiser,M.A.Islam,T.Tanha,andM.S. Zulfiker, “Primary stage of diabetes prediction using machine learning approaches,” in Proc. Int. Conf. on Artificial Intelligence and Smart Systems(ICAIS),Mar.2021,pp.364–367

  8. [16]

    Doexplanationsreflectdecisions?Amachine-centricstrategyto quantify the performance of explainability algorithms,

    Z.Q.Lin,M.J.Shafiee,S.Bochkarev,M.St.Jules,X.Y.Wang,andA. Wong,“Doexplanationsreflectdecisions?Amachine-centricstrategyto quantify the performance of explainability algorithms,” arXiv preprint arXiv:1910.07387,2019

  9. [17]

    The rising burden of non-communicable diseasesinsub-SaharanAfrica,

    J. J. Bigna and J. J. Noubiap, “The rising burden of non-communicable diseasesinsub-SaharanAfrica,”TheLancetGlobalHealth,vol.7,no.10, pp.e1295–e1296,2019

  10. [18]

    Random forests,

    L. Breiman, “Random forests,” Machine Learning, vol. 45, pp. 5–32, 2001

  11. [19]

    Gestationaldiabetesmellitus,

    T.A.Buchanan,A.H.Xiang,etal.,“Gestationaldiabetesmellitus,”*The JournalofClinicalInvestigation*,vol.115,no.3,pp.485–491,2005

  12. [20]

    Managementandpreventionstrategiesfornon-communicable diseases(NCDs)andtheirriskfactors,

    A. Budreviciute, S. Damiati, D. K. Sabir, K. Onder, P. Schuller-Goetzburg, G. Plakys, A. Katileviciute, S. Khoja, and R. Kodzius,“Managementandpreventionstrategiesfornon-communicable diseases(NCDs)andtheirriskfactors,”*FrontiersinPublicHealth*,vol. 8,p.788,2020

  13. [21]

    Handling class imbalance in customer churn prediction,

    J. Burez and D. Van den Poel, “Handling class imbalance in customer churn prediction,” Expert Systems with Applications, vol. 36, no. 3, pp. 4626–4636,2009

  14. [22]

    Aging, diabetes, and the public health system in the United States,

    C.J.Caspersen,G.D.Thomas,L.A.Boseman,G.L.Beckles,andA.L. Albright, “Aging, diabetes, and the public health system in the United States,” *American Journal of Public Health*, vol. 102, no. 8, pp. 1482–1497,2012

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.