Pith. sign in

REVIEW 3 major objections 6 minor 4 references

Diagrams-to-Dynamics (D2D): Exploring Causal Loop Diagram Leverage Points under Uncertainty

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Diagrams-to-Dynamics turns causal loop diagrams into exploratory simulation models, and its leverage-point ranking agrees with a data-calibrated model better than static network centrality does.

desk verdict A genuinely useful low-barrier method for converting CLDs into exploratory linear SDMs, but the headline validation claim is weakened because the prior is tuned using the very benchmark model it is compared against. read the letter →

arxiv 2508.05659 v4 pith:XMPP5Z62 submitted 2025-07-30 cs.LG stat.MEstat.ML

Jeroen F. Uleman , Loes Crielaard , Leonie K. Elsenburg , Guido A. Veldhuis , Naja Hulvej Rod , Rick Quax , Vítor V. Vasconcelos This is my paper · ORCID
classification cs.LGstat.MEstat.ML
keywords causalloopdiagramssystemdynamicsleveragepointsuncertaintyquantificationsensitivityanalysisnetworkcentralityinterventionsimulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Researchers often draw causal loop diagrams (CLDs) to capture feedback in health and environmental problems, but these maps are static and qualitative. This paper proposes Diagrams-to-Dynamics (D2D), a procedure that turns a CLD into an exploratory system dynamics model using only the diagram's structure and the user's labels of variables as stocks, flows/auxiliaries, or constants. D2D samples parameter values from ranges constrained by link polarity, simulates one-standard-deviation interventions on candidate variables, and ranks the effects on an outcome of interest with uncertainty intervals. In a test on a diagram of Alzheimer's disease, D2D's ranking agreed more closely with a data-calibrated system dynamics model than network centrality rankings did, while also providing uncertainty estimates and guidance for future data collection. This makes quantitative leverage-point analysis accessible when no empirical data are available.

What carries the argument

The central object is the polarity-constrained linear system dynamics model derived from a CLD. Its load-bearing pieces are: (1) timescale-separation labeling, which assigns each variable to constants, stocks, or flows/auxiliaries and requires every feedback loop to contain at least one stock; (2) the linear ODE system $d\mathbf{x}/dt = \mathbf{A}\mathbf{x} + \mathbf{b}$, solved analytically by $\mathbf{x}(t) = e^{\mathbf{A}t}\mathbf{x}_0 + (e^{\mathbf{A}t} - I)\mathbf{A}^{-1}\mathbf{b}$; (3) uniform sampling of each unknown link coefficient on the polarity-indicated side of zero, with magnitude bounded by $\theta_{\max}$, a heuristic upper bound derived from the expected total change in the outcome over the timeframe; and (4) repeated simulation of one-standard-deviation interventions, ranking median effects with bootstrap confidence intervals. Together these convert the diagram's polarities and loop structure into a distribution over intervention rankings plus a sensitivity list of the parameters whose measurement would most reduce uncertainty.

What would settle it

On a second CLD where both a D2D model and an independently calibrated SDM exist, set D2D's $\theta_{\max}$ from an external field-based prior rather than from the calibrated model, and check whether D2D's ranking still agrees with the calibrated model better than centrality rankings do; if the advantage disappears, the headline result is an artifact of the shared construction.

Watch

Extended reading notes

Core claim

The central claim is that the structure of an explanatory CLD—node types, link existence, and link polarity—carries enough information to build a provisional quantitative model even when no data are available. D2D labels each variable by timescale: constants change slower than the simulation horizon, stocks accumulate over the base time unit, and flows/auxiliaries adjust instantly; every feedback loop must contain at least one stock. Under a linearity assumption, the resulting equations collapse into a linear system $d\mathbf{x}/dt = \mathbf{A}\mathbf{x} + \mathbf{b}$, which is solved analytically with the matrix exponential. Because link strengths are unknown, each coefficient is drawn from a uniform distribution on the polarity-indicated side of zero, bounded by a heuristic derived from the expected change in the outcome. Repeating draws over many parameter sets produces a median effect and an uncertainty range for each intervention, and a Spearman correlation identifies the parameters whose measurement would most reduce that uncertainty. The validation finding is that D2D's ranking of interventions agreed with a data-calibrated SDM built from the same CLD more closely than did betweenness or closeness centrality rankings.

Load-bearing premise

The central comparison presumes that the calibrated system dynamics model used as the benchmark is independent of D2D, but D2D's parameter bound and equation structure were taken from that same calibrated model and the same CLD, so the reported agreement could instead reflect shared construction choices.

Editorial extensions

If this is right

  • For a given explanatory CLD, D2D provides a ranked list of intervention targets with uncertainty intervals, so even without data a researcher can identify which options are confidently high or low leverage.
  • D2D's sensitivity analysis tells the user which causal links most drive uncertainty in the outcome, effectively recommending where to spend data-collection effort.
  • Because it respects link direction and polarity and simulates all causal paths, D2D avoids the shortest-path and sign-blind assumptions that make centrality-based leverage point inference misleading.
  • D2D can start a Bayesian workflow: the parameter samples act as a prior, and the same model can later be updated with longitudinal data instead of being discarded.
  • The method ships as open-source software with a web interface, so CLD researchers without coding or modeling expertise can run dynamic simulations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, a natural next test is to perturb a CLD's structure—deleting links, adding plausible feedback loops, or re-labeling a stock as an auxiliary—to see how much D2D's rankings change; this would quantify how dependent the conclusions are on the user's labeling decisions.
  • Because D2D's sensitivity analysis names the parameters that matter most for the outcome, the same method could be used prospectively to design measurement plans in new studies, before any data are collected.
  • The Bayesian reading points to a concrete check: once data arrive, updating D2D's uniform priors with the empirical likelihood should shrink the ranking intervals, and a posterior predictive comparison with the calibrated model would show whether D2D's uncertainty was well calibrated.
  • Applying D2D to a CLD built from a participatory process rather than an explanatory model would test the stated boundary condition: if the diagram mixes constructs at different timescales or includes overlapping variables, the ranking intervals should widen or become incoherent, confirming the need for explanatory diagrams.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript proposes Diagrams-to-Dynamics (D2D), a method that converts causal loop diagrams (CLDs) into exploratory linear system dynamics models in the absence of empirical data. The method labels CLD variables as stocks, flows/auxiliaries, or constants; assumes additive linear functional forms; samples link strengths from polarity-constrained uniform distributions; simulates hypothetical interventions; and ranks leverage points by median effect on a variable of interest, with uncertainty estimates and a sensitivity analysis to guide data collection. The method is evaluated on an Alzheimer's disease CLD, where D2D's leverage-point ranking is compared to a calibrated data-driven system dynamics model and to static network centrality analysis. The authors report that D2D aligns more closely with the calibrated model than centrality analysis does and that it provides additional uncertainty information.

Significance. If the central claims are established, D2D fills a real methodological gap: it offers a low-barrier, structurally faithful way to obtain quantitative, dynamic insight from CLDs in data-scarce settings, complementing qualitative analysis and simple centrality metrics. The open-source Python package and web application are concrete, reproducible contributions that will facilitate further testing. The conceptual comparison with network centrality (Table 2) is thoughtful and grounded in existing critiques. However, the empirical validation is currently weakened by (i) a mathematical issue in the stated analytical solution, (ii) a benchmark-dependence in choosing the auxiliary parameter range, and (iii) the absence of quantitative measures for the headline consistency claim. The method itself remains promising, but the evidence presented does not yet fully support the strength of the abstract's conclusions.

major comments (3)
  1. [Section 2.2.2, Eq. (2)] The stated general solution requires the matrix A to be invertible, but no invertibility condition is given. Since the text defines S as the number of stocks and constants, and since constants have zero dynamics, the corresponding rows of A would be zero, making A structurally singular. This makes Eq. (2) invalid as written. Please clarify whether constants are eliminated from the state vector before forming A, or provide the solution form that does not require A^{-1} (e.g., using an integral of the matrix exponential).
  2. [Section 2.2.3 and Section 3.3] The auxiliary θ_max is set to 0.3 in the example because it 'corresponds to the largest empirically estimated auxiliary parameter' from the same calibrated model (Uleman et al. (42)) that is later used as the comparison benchmark. This injects benchmark-derived information into D2D's parameter sampling before the comparison is made, so the reported 'greater consistency with the calibrated model' is not an independent validation of D2D's skill. Please remove this dependence (e.g., by setting θ_max from external literature or a sensitivity analysis) or explicitly justify why this cannot account for the observed rank-order agreement.
  3. [Section 3.3, Figure 4] The claim that D2D 'showed greater consistency with the calibrated model' than centrality analysis is supported only by visual inspection of Figure 4; no quantitative measure (e.g., Spearman rank correlation, top-k overlap, mean absolute rank difference) is reported. Given that D2D and the calibrated model share the same CLD, variable labels, linear equations, and polarity-constrained priors, a quantitative comparison and an ablation study (e.g., varying θ_max or using non-informative priors) are needed to support the headline claim.
minor comments (6)
  1. [Abstract] The phrase 'quantitative CLD analysis methods like network centrality analysis often lead to false inference' is stronger than the evidence in the cited work supports; suggest 'can lead to false inference'.
  2. [Equation (2)] The matrix exponential e^{A t} is not explicitly defined in the text; a brief definition or reference would help readers unfamiliar with matrix exponentials.
  3. [Section 2.2.3] The heuristic formula for θ_max should clarify that 'number of base time units per timeframe' means the total number of simulation steps (T/Δt); the example suggests this, but the wording is ambiguous.
  4. [Figure 4] The panels are described as 'left panel', 'second left panel', etc., which is cumbersome; please add panel labels (a–d) and refer to them by letter in the text.
  5. [Appendix A] The matrix equation in the worked example appears to contain typesetting errors (e.g., repeated subscripts in the matrix entries). Please ensure the notation is typeset correctly.
  6. [Table 2] The rightmost column contains very long sentences; consider splitting the text into concise bullet points for readability.

Circularity Check

1 steps flagged · score 5.0 of 10

D2D's headline comparison to the calibrated SDM is not independent: the auxiliary θ_max is set from that same benchmark model, so the claimed 'greater consistency' is partly constructed rather than out-of-sample evidence.

  1. fitted input called prediction [Section 2.2.3 (Example); comparison in Section 3.3 and Figure 4]
    "For auxiliary variables, we chose a θₘₐₓ of 0.3 because using 0.5 caused most simulations to exceed 10 SDs in the VOI and 0.3 corresponds to the largest empirically estimated auxiliary parameter (i.e., typical standardized regression coefficient) from Uleman et al. (42)."

    The validation benchmark is the data-driven SDM of Uleman et al. (42), and D2D's auxiliary parameter upper bound θₘₐₓ is fixed to the largest fitted coefficient from that same calibrated model. Thus D2D's parameter sampling is scaled using the benchmark's own estimates before D2D is compared with that benchmark. The abstract's claim that 'D2D showed greater consistency with the calibrated model than did static network centrality analysis' is therefore not an independent test: the agreement is partly built into the prior. Section 3.3 acknowledges shared linear equations and polarity-constrained informative priors but does not flag this direct coefficient leakage. Network centrality analysis receives no benchmark-informed tuning, so the Figure 4 comparison is asymmetric.

full rationale

The core D2D conversion from CLD to linear SDM is self-contained: given a labeled CLD, Equation 1 follows from the linearity and polarity assumptions, and the simulation/sensitivity workflow does not depend on fitted data. The circularity is confined to the empirical validation. The auxiliary θₘₐₓ is explicitly chosen as the largest empirically estimated auxiliary parameter from Uleman et al. (42), the same model used as the comparison benchmark, so the 'greater consistency' result is partially constructed from benchmark information. The paper itself notes shared functional forms and polarity-constrained priors in Section 3.3, which further reduces the independence of the comparison, but the direct θₘₐₓ leakage is not mentioned. This is a fitted-input-called-prediction pattern rather than a full self-derivation; the method's internal derivation is not circular. Score 5 reflects partial circularity in the central validation claim, not in the methodology itself.

Assumptions & free parameters 4 free parameters · 7 assumptions · 0 invented entities

The central method introduces no new physical or conceptual entities. It depends on several user-chosen free parameters, strong domain assumptions about CLD completeness and linearity, and an invertibility condition for the analytic solution that is not guaranteed. The validation comparison also assumes an independent benchmark model, which is not actually the case.

free parameters (4)
  • theta_max for stocks = 0.1 in Alzheimer example
    Upper bound for uniform sampling of link strengths into stock equations, chosen so the VOI changes by less than 2 SD over 20 quarters. This is a user-set, scenario-dependent parameter (Section 2.2.3).
  • theta_max for auxiliaries = 0.3 in Alzheimer example
    Upper bound for auxiliary/flow link strengths, chosen as the largest empirically estimated auxiliary coefficient from Uleman et al. (42), the same model used as the comparison benchmark (Section 2.2.3).
  • timeframe and base time unit = 5 years and 3 months in Alzheimer example
    User-chosen temporal scales that determine theta_max and the interpretation of stock coefficients; rankings depend on these choices (Sections 2.2.1 and 2.2.3).
  • N, number of parameter samples = 200 in Alzheimer example
    Number of Monte Carlo parameter samples used for intervention simulations. The paper suggests 50-100 and recommends adjusting until results stabilize, but no formal stopping criterion is given (Section 2.3.1).
assumptions (7)
  • domain assumption The CLD is an accurate and complete representation of the relevant system structure, with no missing variables, links, or feedback mechanisms.
    Stated explicitly as a limitation in Section 3.5; required for D2D results to be meaningful.
  • domain assumption All causal links act as pairwise additive linear terms, with no nonlinearities, delays, or interactions in the default method.
    Stated in Section 2.2.2; interaction terms are optional and not used in the example.
  • domain assumption Variable labeling via timescale separation is correct, and flows/auxiliaries respond instantly while stocks accumulate.
    Section 2.2.1; the paper recommends sensitivity testing labels, but the main results assume the given labels.
  • standard math The linear ODE solution in Equation 2 is valid, which requires the matrix A to be invertible.
    Equation 2 uses A^{-1}; the paper does not state or justify an invertibility condition, and constants included in the state vector may make A singular (Section 2.2.2).
  • domain assumption Uniform sampling from polarity-constrained intervals with user-set theta_max adequately represents epistemic uncertainty in link strengths.
    Section 2.2.3; the choice of uniform priors and the theta_max heuristic is not empirically grounded beyond the single example.
  • domain assumption Interventions can be represented as a one standard deviation shift in a single variable from an average baseline of zero, and effects on the VOI can be compared across variables.
    Section 2.3.1 describes this intervention representation; it is a strong simplification of real-world interventions.
  • domain assumption The data-driven SDM from Uleman et al. (42) is a valid benchmark for evaluating D2D's leverage point ranking.
    Section 3.3 acknowledges that both models share structural assumptions; independence of the benchmark is not established.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Diagrams-to-Dynamics (D2D): Exploring Causal Loop Diagram Leverage Points under Uncertainty." pith.science (2026). https://pith.science/paper/XMPP5Z62

@misc{pith2026250805659,
  author       = {Pith},
  title        = {Pith review of: Diagrams-to-Dynamics (D2D): Exploring Causal Loop Diagram Leverage Points under Uncertainty},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XMPP5Z62}},
  note         = {Machine review of arXiv:2508.05659}
}
read the original abstract

Background: Causal loop diagrams (CLDs) are widely used in health and environmental research to represent hypothesized causal structures underlying complex problems. However, as qualitative and static representations, CLDs are limited in their ability to support dynamic analysis and inform intervention strategies. We propose Diagrams-to-Dynamics (D2D), a method for converting CLDs into exploratory system dynamics models in the absence of empirical data. With minimal user input - following a protocol to label variables as stocks, flows or auxiliaries, and constants - D2D utilizes the structural information already encoded in CLDs, namely the existence and polarity of causal connections, to simulate hypothetical interventions and explore potentially influential places to intervene, known as 'leverage points,' under uncertainty. Results: D2D helps distinguish between high- and low-ranked leverage points. We compare D2D to a calibrated system dynamics model constructed from the same CLD and variable labels. D2D showed greater consistency with the calibrated model than did static network centrality analysis, while also providing uncertainty estimates and guidance for future data collection. Conclusions: The D2D method is implemented in an open-source Python package and a web-based application to support further testing and to lower the barrier to dynamic modeling for researchers working with CLDs. Future studies could help establish the approach's utility across a broad range of cases and domains.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

4 extracted references · 3 canonical work pages

  1. [4]

    Conclusion This paper introduced the D2D approach for generating quantitative insights from CLDs, even without empirical data. D2D enables simulations of hypothetical interventions under uncertainty and offers sensitivity analyses to support early-stage identification of leverage points and help prioritize data collection. Beyond exploration, D2D also ser...

    work page 2022

  2. [15]

    Climate Endgame: Exploring catastrophic climate change scenarios

    Kemp L, Xu C, Depledge J, Ebi KL, Gibbins G, Kohler TA, et al. Climate Endgame: Exploring catastrophic climate change scenarios. Proc Natl Acad Sci. 2022 Aug 23;119(34):e2108146119. 16. Crielaard L, Nicolaou M, Sawyer A, Quax R, Stronks K. Understanding the impact of exposure to adverse socioeconomic conditions on chronic stress from a complexity science ...

    work page 2022

  3. [30]

    Quantifying a systems map: network analysis of a childhood obesity causal loop diagram

    McGlashan J, Johnstone M, Creighton D, de la Haye K, Allender S. Quantifying a systems map: network analysis of a childhood obesity causal loop diagram. PloS One. 2016;11(10):e0165459. 31. Uleman JF, Melis RJ, Quax R, van der Zee EA, Thijssen D, Dresler M, et al. Mapping the multicausality of Alzheimer’s disease through group model building. GeroScience. ...

    work page 2016

  4. [46]

    From,” containing the cause variable, “To,

    Andrade J, Duggan J. A Bayesian approach to calibrate system dynamics models using Hamiltonian Monte Carlo. Syst Dyn Rev. 2021;37(4):283–309. 47. Gelman A, Vehtari A, Simpson D, Margossian CC, Carpenter B, Yao Y, et al. Bayesian Workflow. 2020; Available from: http://arxiv.org/abs/2011.01808 48. Rahmandad H, Akhavan A, Jalali MS. Incorporating Deep Learni...

    arXiv 2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.