{"id":"308c0908-61e4-4c1e-8279-1848799ae314","arxiv_id":"2608.10339","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A causal framework treats Gantt charts of patient journeys as causal DAGs, splits estimation between expert input and observational data, and uses LLMs to scale the expert side.","lead":"The paper introduces expert-guided g-computation, a method that combines clinical expert judgment with data-driven models to estimate how much a hypothetical hospital intervention would reduce average length of stay. The authors add an LLM pipeline to scale expert reasoning and show it matches human expert graphs and time-saving estimates in a safety-net hospital study.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Implemented LLM pipeline outputs deterministic point timestamps, not samples from the expert and timing distributions that Theorem 3.1 requires; reported tau-hat is not covered by the unbiasedness guarantee.","rationale":"The reader's weakest assumption is the unverifiability of Assumption 7: the expert-specified p_expert must equal the true counterfactual waiting-time distribution for direct intervention targets. That is a genuine and important limitation, and the paper's own Section 5 acknowledges it. I found an additional, more internal problem: even granting Assumptions 1 through 7, the implemented LLM pipeline produces deterministic point timestamps rather than draws from p_expert and from the Type II conditional distributions. Theorem 3.1's unbiasedness statement is about the sampling algorithm in Box 3.1; a deterministic plug-in propagated through the recursive max in Eq. (2.3) is not identically distributed as Y(1) unless the relevant conditional distributions are degenerate, which they are not in the paper's own stochastic causal model. Because the headline empirical claim—that egg-computation can rank candidate interventions—rests on the tau-hat values in Tables 8 and 10, this mismatch is load-bearing. The concern is addressable: using stochastic draws or providing a separate unbiasedness proof for the plug-in would resolve it. The reader's verdict of CONDITIONAL remains appropriate; the condition should include reconciling the theoretical sampling requirement with the deployed point-prediction implementation. I therefore recommend UNCHANGED, while noting that the rationale for the conditional verdict is strengthened.","tokens_in":34668,"tokens_out":11254,"duration_ms":115417,"concrete_test":"Run a simulation in the Section 4.1 setting with stochastic waiting times and the correct patient-specific DAGs. Compute tau-hat two ways: (a) oracle egg-computation, sampling from the true conditional distributions of every Type II node and from the true p_expert for Type III nodes, as Theorem 3.1 requires; and (b) the deployed deterministic plug-in, using the true conditional means in place of draws and applying Eq. (2.3) once per patient. If the sampling estimator is unbiased while the plug-in estimator shows bias exceeding its Monte Carlo standard error, the deterministic LLM pipeline is not covered by Theorem 3.1.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Theorem 3.1 guarantees unbiasedness for the sampling procedure in Box 3.1: Type III nodes are drawn from p_expert, Type II nodes from the estimated conditional waiting-time distributions, and the recursion in Eq. (2.3) is applied to those draws, so Y'_i is a draw from the counterfactual law. The deployed pipeline does not implement this. The Type III prompt (Appendix D.1) asks for a single revised timestamp, and the Type II prompt (Appendix D.2) asks for a single delay in hours; Appendix I confirms the comparison runs treat Type II timings deterministically, with each downstream event occurring as soon as its last parent does. Nothing in Section 3.2 or Appendix D describes sampling from the required distributions. This matters even if Assumptions 6 and 7 hold exactly. With parallel branches in the Gantt DAG (Figs. 2 and 6), Y(1) is a max over sums of random waiting times, so a deterministic 'most likely timeline' is neither a draw from the counterfactual distribution nor equal to E[Y(1)|X,D]; the plug-in is subject to Jensen-type bias through the nonlinear max. The paper's own simulation (Table 1) shows nonzero bias for the deterministic DAG-extractor + OLS-timing and kNN-timing variants even with correct patient-specific DAGs, which is consistent with this plug-in distortion. The unverifiability of Assumption 7 is a real concern, but this is a distinct and more proximal problem: even a perfectly calibrated expert distribution would not rescue a deterministic implementation that does not sample from it. The paper should either implement stochastic draws or provide a separate proof that the point-prediction plug-in is unbiased for tau.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes expert-guided g-computation (egg-computation), a causal inference framework for estimating the average time saved by hypothetical operational interventions on hospital length of stay. Each patient's trajectory is represented as a Gantt chart, formalized as a DAG over event waiting times. Under Assumptions 1-7, Theorem 3.1 shows that recursively drawing Type I node times from the factual record, Type II node times from estimated conditional waiting-time distributions, and Type III node times from an expert-supplied counterfactual distribution yields an unbiased estimator of the average treatment effect. The paper then presents an LLM-assisted pipeline to extract patient-specific DAGs and predict event timings at scale. Simulations compare egg-computation with methods that assume a single shared DAG, and a real-world study of eleven candidate interventions at Zuckerberg San Francisco General Hospital reports concordance with human expert annotations, timing-model accuracy, and clinician-reviewed rankings.","tokens_in":34988,"tokens_out":6622,"duration_ms":64603,"significance":"If the ideal sampling procedure described in Theorem 3.1 were actually implemented, egg-computation would be a valuable bridge between operational Gantt-chart reasoning and formal causal inference. The paper has several genuine strengths: a clean theorem with explicit assumptions, a simulation design that isolates structural misspecification, a transparent LLM pipeline with reported costs and validation against human annotators, and a clinically relevant application where the causal and prevalence-based rankings differ. The real-world concordance results, especially the DAG annotation study and the clinician review of counterfactual Gantt charts, are useful practical contributions. The central theoretical guarantee, however, applies to a sampling procedure that the deployed pipeline does not implement. The reported estimates are produced by deterministic point predictions propagated through a nonlinear recursion, so the unbiasedness theorem does not cover the paper's headline numbers. This gap is load-bearing and needs to be addressed before the claims in the abstract and Section 3 can be accepted as stated.","major_comments":[{"comment":"The implemented LLM pipeline does not execute the sampling procedure whose unbiasedness is proved in Theorem 3.1. The Type III prompt in Appendix D.1 asks for a single revised timestamp, the Type II prompt in Appendix D.2 asks for a single delay in hours, and Appendix I states that \"the Type II step is deterministic, with each downstream event occurring as soon as its last parent does.\" No component of the pipeline draws from p_expert or from the estimated conditional waiting-time distributions. Because the recursion in Eq. (2.3) contains a maximum over sums of random waiting times, plugging point predictions into that recursion is not a draw from the counterfactual law and is generally biased for E[Y(1)|X,D]. The paper's own Table 1 is consistent with this concern: DAG-extractor with OLS-timing and kNN-timing shows nonzero bias even when the patient-specific DAG is correct. The manuscript should either implement the required sampling or reformulate the estimator as a deterministic approximation and provide a separate bias analysis for it.","section":"Section 3.2.2, Appendices D.1, D.2, and I"},{"comment":"Assumption 7 requires the expert-specified distribution p_expert to equal the true counterfactual waiting-time distribution for every direct intervention target. The elicitation protocol in Appendix D.1, however, collects only a point timestamp; no distribution is elicited, and no sampling from such a distribution is performed. There is therefore no operational object to which Assumption 7 can attach for the reported estimates. Section 5's call for sensitivity analyses and formal inference for expert uncertainty correctly identifies this as an open issue, but the manuscript still needs to specify how p_expert would be elicited or, alternatively, to state that the deployed estimator targets a different quantity for which Assumption 7 is replaced by a point-calibration assumption.","section":"Section 3.1, Assumption 7, and Appendix D.1"},{"comment":"The leave-one-out MAE evaluation of the Type II timing models does not provide evidence for Assumption 6, which requires equality of the full conditional distribution p(W_k|T_Pa(k),X,D). MAE measures point-prediction accuracy only. A predictor can have low MAE while substantially misrepresenting the conditional distribution, and because the counterfactual recursion in Eq. (2.3) is nonlinear in the waiting times, distributional accuracy is what the unbiasedness theorem requires. The paper should add calibration or coverage diagnostics for the fitted conditional distributions, or replace Assumption 6 and Theorem 3.1 with an explicit statement about what the deterministic pipeline actually estimates.","section":"Section 4.2.2, Tables 3-5"}],"minor_comments":[{"comment":"The boxed algorithm is referred to as \"Box\" without a number; give it a stable label and reference it consistently in the proof of Theorem 3.1 and in Section 3.2.","section":"Section 3, Box 3.1"},{"comment":"The definition \"Nd(k) := Dec(k)\" appears to be a typo for the set of non-descendants; the displayed equation should be corrected.","section":"Section 2, notation"},{"comment":"The abstract and Section 4.2.1 describe the LLM time-saving estimates as \"highly concordant\" with experts, but for disposition-critical imaging the mean difference is 8.1 hours with a 95% CI of [0.0, 24.2], which is substantial relative to the estimated mean savings of 19.6 hours. This caveat should be stated where the concordance claim is made.","section":"Section 4.2.1, Table 2"},{"comment":"Remark A.3 states that Theorem 3.1 requires the Type II timing models to equal the true conditional distributions and that the implementation satisfies this \"to the extent this similarity assumption does.\" This limitation should be acknowledged in the main text in addition to the appendix.","section":"Appendix A, Remark A.3"},{"comment":"The row \"DAG-extractor + Gantt-timing\" achieves zero bias by construction because it uses the true DAG and exact propagation; the text should make explicit that this row does not validate the LLM pipeline's DAG extraction or timing-model components.","section":"Section 4.1, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The disconnect between Theorem 3.1's sampling guarantee and the deployed deterministic LLM pipeline is substantial but correctable. If the authors resubmit with either a proper sampling implementation or an explicit approximate-estimator analysis with bias bounds, the paper could be a strong contribution to the interface of causal inference and LLM-assisted clinical QI. I would not reject solely on the basis of Assumption 7's unverifiability, since the paper properly labels it as an identification assumption; the missing link between theory and implementation is the more serious issue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The formal core is real: Gantt charts get causal semantics through waiting-time DAGs, the Type I/II/III split of expert-versus-data responsibilities is sensible, and Theorem 3.1 is a clean conditional identifiability result, correctly proved under Assumptions 1–7. The second thing is more awkward: the deployed pipeline does not sample from the distributions the theorem requires. The Type III prompt asks for one revised timestamp, the Type II prompt asks for one delay in hours, and Appendix I confirms the Type II step is deterministic. Theorem 3.1 is about drawing Y' from the counterfactual law, not about plugging in point predictions. When parallel branches exist, Y(1) is a max over sums of random waiting times, so a deterministic timeline is neither a draw nor the conditional mean, and Jensen-type bias enters. The paper's own simulation shows this: DAG-extractor plus OLS or kNN timing has nonzero bias even with the correct patient-specific DAG. So the real-world tau-hat numbers are not covered by the unbiasedness guarantee. This is a load-bearing gap between theory and implementation.\n\nCredit where due. The Gantt-as-DAG formalization is new relative to prior LLM causal work and to event-history models; the simulation cleanly separates structural misspecification from timing-model error; the real-world evaluation is unusually transparent about costs, annotation protocol, and disagreement; and the paper openly states Assumption 7 is unverifiable and calls for sensitivity analysis. The related-work discussion is fair and not self-citation heavy. The annotation study is small (24 pairs, five experts) and concordance with experts is not the same as accuracy, but the authors do not oversell it.\n\nThe fixes are clear. Either implement stochastic draws from p_expert and the Type II conditional models, or drop the claim that Theorem 3.1 covers the reported estimates and provide a separate bias analysis for the deterministic plug-in. Since the simulation already contains deterministic variants with nonzero bias, the authors cannot claim the gap is negligible without addressing it. Assumption 7 remains the deeper unverifiable input, but that is acknowledged and is a property of the problem, not a flaw in the proof.\n\nThis paper deserves a serious referee. I would recommend major revision: keep the theory, fix or re-scope the implementation claims, and link the package. I would cite it for the Gantt-DAG formalization and the Type I/II/III split even before revision.","headline":"Clean identifiability theorem, but the deployed LLM pipeline plugs in point predictions rather than sampling, so the reported estimates are not covered by the unbiasedness claim.","tokens_in":35556,"tokens_out":2999,"would_cite":true,"duration_ms":28433,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62D20","68T50"],"pacs":[],"model":"deepseek-v4-flash","headline":"The effect of a not-yet-implemented operational intervention on hospital length of stay can be estimated by combining expert judgment with observed data, allowing hospitals to rank candidate QI interventions by expected time saved before…","keywords":["causal inference","g-computation","Gantt chart","length of stay","large language models","hospital quality improvement","counterfactual timing","expert elicitation"],"falsifier":"Take an intervention that was later implemented, run egg-computation on pre-implementation data, and compare the predicted distribution of counterfactual LOS with the observed post-implementation LOS in an equivalent population; if the predictive interval systematically excludes the observed mean, Assumption 7 or the pooling similarity assumption fails.","tokens_in":34451,"feed_emoji":"🏥","tokens_out":4935,"duration_ms":41883,"temperature":0.7,"pith_summary":"This paper argues that the effect of a not-yet-implemented operational intervention on hospital length of stay can be estimated by combining expert reasoning with observed data: experts specify the causal chart of a patient's events and the direct timing shifts the intervention causes, while statistical models predict downstream timings. The method, egg-computation, treats a Gantt chart as a causal DAG over waiting times and applies g-computation recursively, so the counterfactual discharge time is drawn from the same distribution as the true one under stated assumptions. The paper further shows that LLMs can scale the expert role: in eleven candidate QI interventions at a safety-net hospital, LLM-generated DAGs and time-saving estimates agreed closely with those of human experts, and in-context LLM prediction beat tabular baselines at downstream event timing. The practical payoff is a way to prioritize interventions by expected hours saved even though no patient has ever been exposed to them.","feed_headline":"Untried hospital interventions can be ranked by expected hours saved","feed_subtitle":"Experts draw the causal chart; data predicts downstream waits; at a safety-net hospital, LLM outputs matched human experts.","key_machinery":"The central object is a Gantt chart re-read as a causal DAG: each event is split into start/end nodes, the waiting time $W_k$ is the time from the maximum parent wall time to event $k$, and wall times satisfy $T_k = \\max_{j\\in\\mathrm{Pa}(k)} T_j + W_k$. An exogenous intervention node $A$ attaches to its direct children $\\mathrm{Ch}(A)$, and Theorem 3.1 follows from a local Markov property plus consistency and exclusion restrictions. The machinery is g-computation: in topological order, copy Type I times, draw Type II times from fitted conditional models, and draw Type III times from $p_{\\text{expert}}$; the theorem shows these draws equal the counterfactual distribution.","core_discovery":"Under Assumptions 1 to 7, the paper proves that the counterfactual length of stay generated by egg-computation is identically distributed as $Y_i(1)$, so the estimator $\\hat\\tau$ in (3.6) is unbiased for $\\tau = E[Y(0)-Y(1)]$ (Theorem 3.1). The key division of labor is node type: Type I nodes copy observed times; Type II nodes are imputed by data-driven timing models; Type III nodes, the intervention's direct children, are sampled from an expert-specified distribution $p_{\\text{expert}}$. This is what lets causal claims be made without positivity—no exposed patients—provided the expert-specified DAG and expert counterfactual distributions are correct.","pith_inferences":["A natural extension the paper leaves implicit: if Assumption 7 holds only approximately, sensitivity analyses that vary the direct-effect distribution could bound the ranking error, but the paper does not derive such formal bounds.","Because the pooling step for Type II nodes relies on similarity across patients, a stronger but unstated extrapolation is at work; one could test it by calibrating predictive intervals against held-out delays.","A decisive testable extension would apply egg-computation to a historical intervention that was later implemented, comparing predicted counterfactual LOS with the observed post-implementation LOS in an equivalent population.","The framework's validity is independent of the LLM implementation: hospitals without LLM access could supply $p_{\\text{expert}}$ through structured expert elicitation, and only the scalability, not the identification result, would be lost."],"forward_implications":["Hospitals can rank candidate QI interventions by expected time saved per screened hospitalization before implementing any of them, using the product of the eligibility rate and the estimated average saving.","When patient-specific causal structure is heterogeneous, methods imposing a single population-level DAG can be badly biased; egg-computation avoids this by recovering subject-specific DAGs, with simulation biases dropping from up to about 14 hours to near zero.","An LLM-assisted pipeline can produce DAGs that human experts largely endorse: intervention-target specificity near 96.5%, edge precision near 92-94%, and edge recall around 80-96% across two interventions, with 19 of 24 pairs yielding identical time-saved estimates.","In-context LLM prediction of downstream event timings achieves lower mean absolute error (24.1 hours) than tabular baselines (best baseline 27.2 hours), especially for longer waits.","The framework extends beyond healthcare to any time-ordered process whose task dependencies can be drawn as Gantt charts, such as manufacturing, software development, and human-resource workflows."],"supporting_citations":[{"why":"Supplies the g-computation recursion and the single-world causal semantics used to prove identification.","marker":"Robins (1986)"},{"why":"Provides the iterative graph expansion procedure the LLM prompts encode for confounder search.","marker":"Guo and Zhao (2026)"},{"why":"Gives the disjunctive cause criterion used to decide which pre-exposure variables to retain in the DAG.","marker":"VanderWeele (2019)"},{"why":"Prior bottleneck analysis that generated the eleven candidate interventions and the clinical-note data pipeline.","marker":"Vossler et al. (2026)"},{"why":"Introduces in-context learning, the mechanism behind the LLM-based Type II timing predictions.","marker":"Brown et al. (2020)"},{"why":"Evidence that LLMs can extract small causal DAGs, motivating the multi-step DAG construction pipeline.","marker":"Kıcıman et al. (2024)"}],"fun_headline_variants":["Rank untried hospital fixes by predicted hours saved with AI+experts","Expert-guided AI predicts time savings for untried hospital changes","LLM-assisted causal charts estimate hours saved for untried interventions","For untried interventions, expert-AI combo estimates time saved"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is Assumption 7: the expert-supplied distribution of the intervention's direct target timings equals the true counterfactual waiting-time distribution, a quantity that is structurally unobserved and can never be checked against data.","fun_headline_variants_meta":{"raw":{"variants":["Rank untried hospital fixes by predicted hours saved with AI+experts","Expert-guided AI predicts time savings for untried hospital changes","LLM-assisted causal charts estimate hours saved for untried interventions","For untried interventions, expert-AI combo estimates time saved"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001488,"raw_usage":{"total_tokens":5995,"prompt_tokens":985,"completion_tokens":5010,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":601,"completion_tokens_details":{"reasoning_tokens":4938}},"tokens_in":601,"tokens_out":5010,"duration_ms":28317,"temperature":1.0,"reasoning_tokens":4938,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:21:54.040722+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take an intervention that was later implemented, run egg-computation on pre-implementation data, and compare the predicted distribution of counterfactual LOS with the observed post-implementation LOS in an equivalent population; if the predictive interval systematically excludes the observed mean, Assumption 7 or the pooling similarity assumption fails.","supporting_citations":[{"cited_title":"A new approach to causal inference in mortality studies with a sustained exposure period---application to control of the healthy worker survivor effect","cited_arxiv_id":null,"evidence_quote":"Supplies the g-computation recursion and the single-world causal semantics used to prove identification."},{"cited_title":"Confounder selection via iterative graph expansion","cited_arxiv_id":null,"evidence_quote":"Provides the iterative graph expansion procedure the LLM prompts encode for confounder search."},{"cited_title":"Principles of confounder selection","cited_arxiv_id":null,"evidence_quote":"Gives the disjunctive cause criterion used to decide which pre-exposure variables to retain in the DAG."}],"review_version":1}