{"id":"e3c1be5f-aa52-4b44-8c6c-a4f392d24c64","arxiv_id":"2411.14939","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"An XGBoost model that predicts platelet request returns, combined with a 'youngest unit for predicted returns' issuing policy, is estimated to reduce simulated platelet wastage by 8 to 14 percent without reducing service level.","lead":"A hospital blood bank study trains a machine learning model to predict which platelet requests will be returned unused, then tests a 'youngest unit for predicted returns' issuing policy in a simulation of the blood bank. Simulated results suggest platelet wastage could fall by roughly 8 to 14 percent without lowering service levels, with larger gains where units arrive with less shelf life.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The advertised 14% wastage reduction is not the number from the paper's own real-demand evaluation, which shows a smaller UCLH benefit (1.1% vs 1.2%) and no confidence interval.","rationale":"The paper is careful and unusually honest: it reports two evaluation routes, acknowledges the single-unit/independent-label simplification in Section 4.3.3 and the 2017 discrepancy in Supplementary Note J, and releases code. My concern is not that the policy is worthless; the real-demand evaluation supports the direction of the effect, and the US-style setting shows a 14% relative reduction. The load-bearing problem is that the most prominent number in the abstract and Results is taken from the less realistic contour-lookup route, while the more realistic route gives a smaller UCLH effect and no uncertainty quantification. Since the central claim includes a specific magnitude ('14%'), this is a reporting and robustness concern, not a soundness failure. Threshold selection from the training-set simulation is defensible because the test set remains separate, but it does not remove the need to report the real-demand estimate as primary. I therefore keep the reader's CONDITIONAL verdict: the qualitative direction is credible, but the abstract should be revised to report the real-demand evaluation with confidence intervals and treat 14% as an idealized-simulation result.","tokens_in":30234,"tokens_out":5144,"duration_ms":51204,"concrete_test":"Recompute the primary benefit using the Supplementary Note J real-demand evaluation rather than contour interpolation: bootstrap the 2017 request/return records (e.g., resample days with replacement for 1,000 replicates) for both the UCLH and R&R shelf-life settings, and report 95% confidence intervals for wastage and service level under YUPR versus OUFO. If the UCLH confidence interval for the wastage reduction includes zero, the abstract should state a direction-only benefit and present the 14% figure as an explicitly labeled idealized-simulation sensitivity analysis.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline quantitative claim in the abstract and Section 2.2 ('reduce wastage by 14% ... from 0.91% to 0.78%') is generated by looking up test-set sensitivity/specificity on contour plots from Experiment 2, with the threshold chosen to minimize wastage on the training set (Section 4.2.2). Those contours are produced by Algorithm 1, which assumes every request is for a single unit, samples the true return label as an independent Bernoulli(rho), and simulates predictions from a constant (sensitivity, specificity) with no dependence on ward, timing, platelet count, or stock state. The paper's own more realistic route, Supplementary Note J, plugs real 2017 requests and actual model predictions into the same workflow. It gives 1.1% vs 1.2% wastage at UCLH, an 8.3% relative reduction rather than the advertised 14%, and 4.3% vs 5.0% under the R&R shelf-life distribution. No confidence interval is reported for these one-year point estimates, so the small UCLH difference may be within sampling noise. The direction of the effect is consistent across routes, which supports the qualitative policy claim, but the magnitude in the abstract is not robust to the paper's own more realistic evaluation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an ML-guided platelet issuing policy (YUPR: youngest unit first for predicted returns) and evaluates it at a single UK hospital trust. An XGBoost model is trained on 17,297 UCLH platelet requests from 2015-2016 and tested on 9,353 requests from 2017, achieving AUROC 0.74. The authors use a simulation-first approach: they build a discrete-event simulation of the blood bank workflow with returns, generate wastage/service-level contour plots over sensitivity/specificity pairs, and then use the test-set sensitivity/specificity at a threshold chosen on the training set to claim a 14% wastage reduction (0.91% to 0.78%) with no service-level detriment. They also provide a complementary real-demand evaluation in Supplementary Note J that plugs actual 2017 requests and model predictions into the same workflow, reporting UCLH wastage 1.2% to 1.1% and R&R shelf-life wastage 5.0% to 4.3%, again with no service-level reduction. Sensitivity analyses show that the potential benefit increases with higher return rates and shorter remaining shelf life on arrival.","tokens_in":30538,"tokens_out":5814,"duration_ms":55721,"significance":"If the qualitative claim holds, this is a useful contribution to blood inventory management: it identifies post-issue returns as a largely neglected factor, proposes a concrete issuing policy that targets returns, and demonstrates that even an imperfect predictive model can reduce wastage while maintaining service level. The core ML evaluation is not circular: the model is trained on independent historical labels and evaluated on a temporal holdout. Additional strengths include the simulation-first methodology, the use of KPI-based evaluation rather than only AUROC, the inclusion of real-demand evaluation in the supplement, and the provided code and non-identifiable simulation inputs. The main weakness is that the headline quantitative claim is taken from an idealized contour-lookup procedure rather than from the paper's own more realistic real-demand evaluation, and that realistic evaluation is reported without uncertainty. The direction of the effect is consistent across both evaluation routes, but the magnitude in the abstract is not robust to the paper's more realistic route.","major_comments":[{"comment":"The abstract and Section 2.2 headline a 14% reduction in wastage (0.91% to 0.78%) for the UCLH scenario. This figure is obtained by reading the test-set sensitivity/specificity off contour plots produced under the simplifying assumptions of Section 4.3.3 (single-unit requests, independent Bernoulli return labels, fixed sensitivity/specificity). The paper's own real-demand evaluation in Supplementary Note J gives UCLH wastage 1.2% under OUFO and 1.1% under the ML-guided policy, i.e., an 8.3% relative reduction rather than 14%, and 5.0% to 4.3% under R&R shelf life; no confidence interval is reported for either one-year point estimate. This discrepancy is material because the abstract presents 14% as the headline estimate. The authors should either make the Note J result the primary estimate with an uncertainty quantification, or clearly label the 14% figure as an idealized single-unit simulation result, not the estimated real-world benefit.","section":"Abstract; Section 2.2; Supplementary Note J"},{"comment":"The classification threshold is selected by minimizing estimated wastage on the training set using the same contour plots from Experiments 2 and 4, and the test-set sensitivity/specificity at that threshold are then looked up on those same contours. Because the contours are generated with inputs from UCLH 2015-2016 (or R&R), the reported KPI is an in-sample estimate of an optimized operating point and has no uncertainty estimate. A reader cannot tell whether the 1.1% versus 1.2% difference observed in Supplementary Note J is within sampling noise. Please provide a confidence interval or bootstrap for the real-demand KPI simulation and, if possible, a pre-registered or less simulation-dependent threshold selection rule.","section":"Section 4.2.2"},{"comment":"Algorithm 1 assumes that the true return label is an independent Bernoulli(rho) draw and that the model's prediction depends only on the true label through fixed sensitivity and specificity. This removes any correlation between returns, model errors, and request features such as ward, platelet count, or stock state, correlations that the SHAP analysis itself shows to be important. The manuscript acknowledges the single-unit discrepancy in Section 4.2.2 and supplies Supplementary Note J as a corrective, but the corrected route is a single one-year point estimate. To make the headline claim load-bearing, the authors should quantify the gap between the two routes, for example by running the real-demand workflow over multiple years or by reporting both the idealized and real-demand results with appropriate caveats.","section":"Section 4.3.3; Algorithm 1"}],"minor_comments":[{"comment":"The caption says 'combinations of sensitivity and sensitivity' and should read 'sensitivity and specificity'.","section":"Figure 1 caption"},{"comment":"There are duplicated words: 'using a a standing order' in Section 4.3.4 and 'on the the daily cost' in the Figure E2 caption.","section":"Section 4.3.4 and Figure E2 caption"},{"comment":"The Discussion says 'impact on a patent's health' and should say 'patient's health'; Section 4.2.2 contains 'a region of the of the curve'.","section":"Discussion and Section 4.2.2"},{"comment":"The statement 'no detriment to service level' is supported by the real-demand Note J numbers (99.4% vs 99.4% and 99.1% vs 99.1%), but the contour-interpolation route used for the headline 14% figure does not report a service-level value at the selected operating point; clarifying which route supports each claim would improve transparency.","section":"Section 2.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is in scope for the journal and the underlying modeling is reproducible in principle. My recommendation is driven by a framing issue rather than by a defect in the ML methodology: the headline 14% figure is not the number produced by the paper's own most realistic evaluation. If the authors re-center the paper on the real-demand result, or clearly re-frame the 14% as an idealized simulation result with the real-demand number as the primary estimate, and add uncertainty quantification for that estimate, the contribution would be solid. I see no grounds for rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The paper is the first to treat post-issue platelet returns as a decision variable, and it proposes a simple, plausible policy: issue the freshest units when the model predicts a return. The ML result itself is solid—AUROC 0.74 on a temporal holdout—and the authors are honest about limitations. But the headline 14% wastage reduction is not the number from their own more realistic evaluation. Their real-demand simulation gives 1.1% vs 1.2% at UCLH, about an 8% relative reduction rather than 14%, and no confidence interval. For the US shelf-life scenario the real-demand comparison is 4.3% vs 5.0%, which is more meaningful but still not the headline figure.\n\nWhat is genuinely new: the YUPR policy and the return-prediction model. The simulation-first approach—mapping sensitivity/specificity onto wastage contours before training—is a good example of thinking about operational utility rather than just AUC. They also give reasonable credit to related work and clearly flag the single-type and no-compatibility assumptions, plus staff-adherence concerns. Code is on GitHub; the underlying patient data are not public, but the simulation inputs are in the supplement.\n\nSoft spots, in order of seriousness. First, the 14% figure comes from contour plots built on the assumption of single-unit requests, independent Bernoulli return labels, and a model characterized purely by constant sensitivity and specificity. That is a simplification, and the authors know it, which is why they built the real-demand evaluation. The real-demand number should be the headline. Second, the classification threshold is chosen on the training set using the same simulation contours, so the reported test-set benefit is a bit optimistic; not circular, but not neutral either. Third, at UCLH the absolute wastage is under 1%, so the absolute saving is tiny; the US scenario is more compelling but speculative. The direction of the effect is consistent across all routes, so the qualitative claim—returns-aware issuing can reduce wastage without hurting service level—holds up.\n\nWho this is for: people working on perishable inventory or operational ML in hospitals. It deserves serious peer review; the novelty and the thoroughness warrant referee time. I would ask the authors to move the real-demand evaluation to the abstract and add uncertainty around the point estimates.","headline":"Novel returns-aware platelet issuing policy, but the advertised 14% saving comes from an idealized simulation; the paper's own real-demand evaluation shows a smaller, still positive effect.","tokens_in":31074,"tokens_out":2122,"would_cite":true,"duration_ms":22937,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T05","90B05"],"pacs":[],"model":"deepseek-v4-flash","headline":"A machine-learning model that predicts which platelet requests will be returned can guide hospital issuing decisions to cut waste without lowering service levels.","keywords":["platelet inventory","issuing policy","machine learning","returns","wastage reduction","simulation-first evaluation","hospital blood bank","gradient-boosted trees"],"falsifier":"Run YUPR against Oldest-Unit-First-Out prospectively at a hospital with a return rate around 8% and measure wastage and service level over a full year; if wastage is not lower, or the service level falls, the central claim fails. The paper's own real-demand simulation already provides a partial check, showing smaller gains (1.1% versus 1.2% waste at the partner hospital) than the 14% headline figure.","tokens_in":30043,"feed_emoji":"🩸","tokens_out":5435,"duration_ms":53308,"temperature":0.7,"pith_summary":"This paper argues that the routine practice of issuing the oldest platelet unit first becomes suboptimal when some issued units are returned unused, and that a machine-learning model can exploit returns to reduce waste. It trains a model on 17,297 platelet requests to predict, for each request, whether at least one issued unit will come back, reaching an AUROC of 0.74 on 9,353 held-out requests. Embedding that model in a simulation of a hospital blood bank, the paper estimates that a policy of issuing the freshest units when returns are predicted and the oldest otherwise would cut wastage by about 14% (from 0.91% to 0.78%) at the partner hospital without lowering the service level. The estimated benefit grows when return rates are higher or remaining shelf life on arrival is shorter, as in a US-style scenario where wastage falls from 5.0% to 4.3% in the real-demand evaluation.","feed_headline":"Predicting platelet returns cuts blood-bank waste 14%","feed_subtitle":"ML-guided issuing keeps service levels while saving platelets; gains grow where shelf life is short.","key_machinery":"The load-bearing mechanism is the YUPR issuing policy combined with a pre-deployment simulation that maps combinations of model sensitivity and specificity to wastage and service level. The simulation models daily ordering, morning and afternoon demand, returns at midday, slippage (units returned in a state that cannot be reissued), and expiry, with parameters estimated from hospital data. A single-unit demand assumption lets each request's true return label be drawn as a Bernoulli($\\rho$) and the predicted label from fixed sensitivity $\\alpha$ and specificity $\\beta$, producing contour plots that define the region where YUPR beats OUFO; the trained model's ROC curve is then superimposed to read off the wastage reduction. The ML model itself is a gradient-boosted decision tree trained with features such as recent platelet count and how far in advance the request was made.","core_discovery":"The central claim is that platelet returns are predictable enough to change issuing decisions, and that acting on the prediction improves the blood bank's key performance indicators. The paper proposes Youngest Unit for Predicted Returns (YUPR): if the model predicts a request will not be transfused, issue the freshest unit so that a returned unit has maximum time left before expiry; otherwise issue the oldest unit as usual. In the simulation, this policy beats Oldest-Unit-First-Out on wastage with no service-level loss, and the advantage is largest with higher return rates and shorter remaining shelf life. The paper also claims a methodological point: a simulation-first evaluation can show whether a predictive model has operational value before the model is built, and can guide the performance requirements for the model.","pith_inferences":["The paper's mechanism is not limited to platelets: the same 'issue freshest when return is likely' logic could apply to red blood cells or other perishable medical stock where units are returned and reissued, though the paper only studies platelets.","The paper's simulation treats model quality by fixed sensitivity and specificity with independent return labels, so a richer evaluation that lets return probability depend on ward, timing, platelet count, and stock state would likely change the estimated benefit; the paper's own real-demand simulation shows a smaller benefit at the partner hospital (1.1% versus 1.2% waste).","Combining return predictions with replenishment decisions, such as ordering less when predicted returns are high, could reduce waste further; the paper notes this is not tested.","The wastage-minimising classification threshold depends on the operating scenario, so hospitals adopting the policy would need to recalibrate the threshold locally."],"forward_implications":["Hospitals can lower platelet wastage without reducing service by switching issuing from oldest-first to model-guided.","The policy matters most where return rates are high and units arrive with less remaining shelf life, because those settings show larger absolute reductions.","Simulation-first evaluation can be used to decide whether to invest in ML for inventory workflows before building models, and to set target performance levels.","A rule-based or simpler predictor based on the model's insights might capture part of the benefit, since the partner hospital is exploring that route.","The benefits require the ability to reissue returned units; sites where returned units are not in reissuable state see no gain."],"supporting_citations":[{"why":"Supplies the alternative US distribution of remaining useful life on arrival and the cost parameters used in the simulation, creating the key 'shorter shelf life' scenario where YUPR's benefit is larger.","marker":"[73]"},{"why":"Systematic review documenting common platelet wastage rates and the short shelf life, establishing the problem the paper addresses.","marker":"[1]"},{"why":"Previous platelet inventory simulation work that the paper's simulated workflow builds on.","marker":"[61]"},{"why":"Supplies the gradient-boosted decision tree method used to train the predictive model.","marker":"[86]"},{"why":"Recent work using simulation to evaluate trained ML models in healthcare workflows, which the paper extends with its simulation-first approach.","marker":"[53]"},{"why":"Closest prior proposal to assign older units to patients most likely to be transfused, providing the antecedent for likelihood-informed issuing.","marker":"[48]"}],"fun_headline_variants":["ML-guided platelet issuing cuts waste 14%","AI predicts platelet returns, trims waste 14%","Smarter issuing: machine learning cuts platelet waste","ML model reduces blood-bank platelet waste 14%","Predicting returns: ML cuts platelet waste 14%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline 14% reduction comes from contours built on the assumption that a model is fully captured by fixed sensitivity and specificity and that returns happen independently with a fixed probability; if returns and model errors correlate with wards, timing, platelet counts, or stock state, the contour estimate can misstate the true benefit.","fun_headline_variants_meta":{"raw":{"variants":["ML-guided platelet issuing cuts waste 14%","AI predicts platelet returns, trims waste 14%","Smarter issuing: machine learning cuts platelet waste","ML model reduces blood-bank platelet waste 14%","Predicting returns: ML cuts platelet waste 14%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000141,"raw_usage":{"total_tokens":1113,"prompt_tokens":840,"completion_tokens":273,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":456,"completion_tokens_details":{"reasoning_tokens":196}},"tokens_in":456,"tokens_out":273,"duration_ms":3225,"temperature":1.0,"reasoning_tokens":196,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:41:49.840906+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run YUPR against Oldest-Unit-First-Out prospectively at a hospital with a return rate around 8% and measure wastage and service level over a full year; if wastage is not lower, or the service level falls, the central claim fails. The paper's own real-demand simulation already provides a partial check, showing smaller gains (1.1% versus 1.2% waste at the partner hospital) than the 14% headline figure.","supporting_citations":[{"cited_title":"& Ravindran, A","cited_arxiv_id":null,"evidence_quote":"Supplies the alternative US distribution of remaining useful life on arrival and the cost parameters used in the simulation, creating the key 'shorter shelf life' scenario where YUPR's benefit is larger."},{"cited_title":"& Srinivas, S","cited_arxiv_id":null,"evidence_quote":"Previous platelet inventory simulation work that the paper's simulated workflow builds on."},{"cited_title":"& Guestrin, C","cited_arxiv_id":null,"evidence_quote":"Supplies the gradient-boosted decision tree method used to train the predictive model."},{"cited_title":"& Shah, N","cited_arxiv_id":null,"evidence_quote":"Recent work using simulation to evaluate trained ML models in healthcare workflows, which the paper extends with its simulation-first approach."},{"cited_title":"& Prastacos, G","cited_arxiv_id":null,"evidence_quote":"Closest prior proposal to assign older units to patients most likely to be transfused, providing the antecedent for likelihood-informed issuing."}],"review_version":1}