pith. sign in

arxiv: 2205.00093 · v2 · submitted 2022-04-29 · 📊 stat.ME · stat.AP

Bayesian Benefit-Risk Assessment with Dependent Outcomes via Latent Factor Models

Konstantinos Vamvourellis , Konstantinos Kalogeropoulos , Lawrence Phillips This is my paper

Pith reviewed 2026-05-24 11:21 UTC · model grok-4.3

classification 📊 stat.ME stat.AP
keywords benefit-risk analysislatent factor modelsBayesian methodssequential Monte CarloMCDAmixed outcomessequential decision makingdrug assessment
0
0 comments X

The pith

Bayesian latent factor models handle dependent mixed outcomes in drug benefit-risk analysis and enable sequential MCDA score updates.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper constructs a Bayesian framework that combines multiple benefit and risk criteria while explicitly modeling correlations among mixed outcome types through extended latent factor structures. Model choice integrates in-sample fit with out-of-sample predictive accuracy, and sequential Monte Carlo algorithms update the resulting MCDA scores as fresh observations arrive. This setup permits comparisons between treatments to shift over time and supports early stopping rules or adaptive allocation. Readers should care because conventional benefit-risk tools often assume outcome independence and deliver only static snapshots, limiting their use in ongoing trials or regulatory reviews.

Core claim

We develop a coherent Bayesian framework for benefit-risk analysis that addresses these challenges and supports sequential decision-making. We extend structured factor models to mixed outcomes and introduce a principled approach for selecting among competing specifications that combines model fit with out-of-sample predictive performance. We then develop a sequential estimation framework that updates MCDA scores as new data become available, allowing treatment comparisons to evolve over time. This supports early stopping when conclusions are clear and permits dynamic treatment allocation aligned with study objectives. To make this feasible, we develop tailored sequential Monte Carlo methods.

What carries the argument

Extended structured latent factor models for mixed outcomes that induce dependence among binary, continuous and count endpoints and feed directly into time-updated MCDA scores.

If this is right

  • MCDA scores and treatment rankings can be recomputed in real time as patient data accumulate.
  • Trials can stop early once posterior probabilities of superiority stabilize.
  • Treatment allocation probabilities can shift dynamically toward arms favored by current benefit-risk estimates.
  • Competing factor specifications can be compared on both goodness-of-fit and predictive criteria before final inference.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same sequential updating structure could be applied to multi-criteria decisions outside drug development, such as device approval or health-technology assessment.
  • If the latent-factor assumption holds only approximately, hybrid models that blend factor dependence with residual copulas might improve robustness.
  • The tailored SMC samplers may generalize to other Bayesian models with mixed data and sequential arrival of observations.

Load-bearing premise

The latent factor structure can adequately capture dependencies between mixed outcomes without substantial misspecification.

What would settle it

A hold-out validation on the diabetes trial data in which predictions from the selected factor model show materially worse calibration or ranking accuracy than an independence baseline.

Figures

Figures reproduced from arXiv: 2205.00093 by Konstantinos Kalogeropoulos, Konstantinos Vamvourellis, Lawrence Phillips.

Figure 1
Figure 1. Figure 1: MCDA Scores at the end of the sequential run. Avandia (AVM) scores substantially higher than the other [PITH_FULL_IMAGE:figures/full_fig_p025_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Sequentially updated probabilities of P(S(AVM) > S(MET)) and P(S(AVM) > S(RSG)). We see that the probabilities converge to 1 within the first 300 patients. A dynamic trial could either have concluded early or assigned the remaining patients to AVM given that it is considered a better treatment based on MCDA scores. The lines indicate that AVM (in blue) is shown to have a higher predicted score early on, wi… view at source ↗
Figure 3
Figure 3. Figure 3: Sequentially updated probabilities of P(S(AVM) > S(MET)) and P(S(AVM) > S(RSG)). We see that the probabilities converge to 1 within the first 300 patients. A dynamic trial could have concluded early based on this evidence or could have assigned the remaining patients to AVM given that it is considered a better treatment based on MCDA scores. MCDA is a general framework that can be used in association with … view at source ↗
Figure 4
Figure 4. Figure 4: Sequentially updated posterior mean (lines) and the 95% central quantiles (shade bands) of the posterior [PITH_FULL_IMAGE:figures/full_fig_p029_4.png] view at source ↗
read the original abstract

Approving and assessing new drugs is complex because multiple criteria must be considered simultaneously. A common approach is benefit-risk analysis, often conducted within a Bayesian framework to account for uncertainty and combine data with expert judgement, typically through multi-criteria decision analysis (MCDA) scores. This requires models that accommodate mixed and potentially correlated outcomes; latent factor models provide a natural framework. We develop a coherent Bayesian framework for benefit-risk analysis that addresses these challenges and supports sequential decision-making. We extend structured factor models to mixed outcomes and introduce a principled approach for selecting among competing specifications that combines model fit with out-of-sample predictive performance. We then develop a sequential estimation framework that updates MCDA scores as new data become available, allowing treatment comparisons to evolve over time. This supports early stopping when conclusions are clear and permits dynamic treatment allocation aligned with study objectives. To make this feasible, we develop tailored sequential Monte Carlo methods adapted to the model structure. The methodology is illustrated using data on patients with type II diabetes treated with Metformin, Rosiglitazone, and their combination.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The manuscript develops a Bayesian latent factor model framework for benefit-risk analysis of treatments with mixed, potentially dependent outcomes. It extends structured factor models to mixed data types, proposes a model selection criterion that balances in-sample fit with out-of-sample predictive performance, and introduces tailored sequential Monte Carlo methods to enable dynamic updating of multi-criteria decision analysis (MCDA) scores as new data arrive, supporting early stopping and adaptive allocation. The approach is illustrated on type II diabetes data comparing Metformin, Rosiglitazone, and their combination.

Significance. If the latent factor structure and SMC implementation perform as intended, the work supplies a coherent, uncertainty-aware approach to benefit-risk assessment that properly accounts for outcome dependencies and enables sequential decision-making. This addresses a practical need in drug evaluation and regulatory contexts. The combination of model selection with predictive criteria and the adaptation of SMC to the factor model structure are methodological strengths.

minor comments (2)
  1. [Abstract] Abstract: the description of the diabetes illustration does not report any quantitative results (e.g., selected model, estimated dependencies, or changes in MCDA scores over time), making it difficult to assess the practical impact of the proposed methods.
  2. The manuscript would benefit from an explicit statement of the precise form of the mixed-outcome likelihood and the identifiability constraints imposed on the latent factor loadings.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive summary of the manuscript, recognition of its methodological contributions, and recommendation for minor revision. No specific major comments were provided in the report.

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper extends structured latent factor models to mixed outcomes, proposes a model-selection criterion blending fit and out-of-sample prediction, and develops tailored SMC for sequential MCDA updating. No equations or steps reduce by construction to fitted parameters renamed as predictions, self-definitional loops, or load-bearing self-citations whose content is unverified. The framework is presented as a coherent Bayesian extension relying on standard factor-model and SMC techniques whose assumptions are stated explicitly rather than smuggled in. The derivation chain remains self-contained against external benchmarks and does not invoke uniqueness theorems or ansatzes from prior author work as the sole justification.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Only the abstract is available, so the full ledger cannot be extracted. The approach likely depends on standard Bayesian priors and assumptions about factor model structure for mixed data.

axioms (1)
  • domain assumption Latent factor models can capture dependencies in mixed outcomes
    The framework relies on this to handle correlated data.

pith-pipeline@v0.9.0 · 5719 in / 1057 out tokens · 25182 ms · 2026-05-24T11:21:50.680791+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

51 extracted references · 51 canonical work pages

  1. [1]

    Doucet, and R

    Andrieu, C., A. Doucet, and R. Holenstein (2010). Particle markov chain monte carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology)\/ 72\/ (3), 269--342

    work page 2010

  2. [2]

    Andrieu, C. and G. O. Roberts (2009). The pseudo-marginal approach for efficient monte carlo computations. The Annals of Statistics\/ 37\/ (2), 697--725

    work page 2009

  3. [3]

    Blei, D. M., A. Kucukelbir, and J. D. McAuliffe (2017). Variational inference: A review for statisticians. Journal of the American statistical Association\/ 112\/ (518), 859--877

    work page 2017

  4. [4]

    Demko, G

    Carlisle, B., N. Demko, G. Freeman, A. Hakala, N. MacKinnon, T. Ramsay, S. Hey, A. J. London, and J. Kimmelman (2016). Benefit, risk, and outcomes in drug development: a systematic review of sunitinib. JNCI: Journal of the National Cancer Institute\/ 108\/ (1)

    work page 2016

  5. [5]

    Gelman, M

    Carpenter, B., A. Gelman, M. D. Hoffman, D. Lee, B. Goodrich, M. Betancourt, M. Brubaker, J. Guo, P. Li, and A. Riddell (2017). Stan: A probabilistic programming language. Journal of statistical software\/ 76\/ (1)

    work page 2017

  6. [6]

    (2002, aug)

    Chopin, N. (2002, aug). A sequential particle filter method for static models . Biometrika\/ 89\/ (3), 539--552

    work page 2002

  7. [7]

    (2004, dec)

    Chopin, N. (2004, dec). Central limit theorem for sequential Monte Carlo methods and its application to Bayesian inference . The Annals of Statistics\/ 32\/ (6), 2385--2411

    work page 2004

  8. [8]

    Chopin, N., P. E. Jacob, and O. Papaspiliopoulos (2012, oct). SMC 2: an efficient algorithm for sequential analysis of state space models . Journal of the Royal Statistical Society: Series B (Statistical Methodology)\/ 75\/ (3), 397--426

    work page 2012

  9. [9]

    Fr \"u hwirth-Schnatter, J

    Conti, G., S. Fr \"u hwirth-Schnatter, J. J. Heckman, and R. Piatek (2014). Bayesian exploratory factor analysis. Journal of Econometrics\/ 183\/ (1), 31--57

    work page 2014

  10. [10]

    Costa, M. J., W. He, Y. Jemiai, Y. Zhao, and C. Di Casoli (2017). The case for a bayesian approach to benefit-risk assessment: overview and future directions. Therapeutic innovation & regulatory science\/ 51\/ (5), 568--574

    work page 2017

  11. [11]

    Dai, C., J. Heng, P. E. Jacob, and N. Whiteley (2020). An invitation to sequential monte carlo samplers

    work page 2020

  12. [12]

    Dawid, A. P. and M. Musio (2015). Bayesian model selection based on proper scoring rules. Bayesian analysis\/ 10\/ (2), 479--499

    work page 2015

  13. [13]

    Doucet, and A

    Del Moral, P., A. Doucet, and A. Jasra (2006). Sequential monte carlo samplers. Journal of the Royal Statistical Society: Series B (Statistical Methodology)\/ 68\/ (3), 411--436

    work page 2006

  14. [14]

    Dodgson, J. S., M. Spackman, A. Pearman, and L. D. Phillips (2009). Multi-criteria analysis: a manual

    work page 2009

  15. [15]

    Fr \"u hwirth-Schnatter, S. and H. F. Lopes (2018). Parsimonious Bayesian factor analysis when the number of factors is unknown. Unpublished Working Paper\/

    work page 2018

  16. [16]

    Garrison Jr, L. P., A. Towse, and B. W. Bresnahan (2007). Assessing a structured, quantitative health outcomes approach to drug risk-benefit analysis. Health Affairs\/ 26\/ (3), 684--695

    work page 2007

  17. [17]

    Ghosh, J. and D. B. Dunson (2009). Default prior distributions and efficient posterior computation in Bayesian factor analysis. Journal of Computational and Graphical Statistics\/ 18\/ (2), 306--320

    work page 2009

  18. [18]

    Glasziou, P. P. and L. M. Irwig (1995). An evidence based approach to individualising treatment. Bmj\/ 311\/ (7016), 1356--1359

    work page 1995

  19. [19]

    Gneiting, T. and A. E. Raftery (2007). Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association\/ 102\/ (477), 359--378

    work page 2007

  20. [20]

    Guo, J. J., S. Pandey, J. Doyle, B. Bian, Y. Lis, and D. W. Raisch (2010). A review of quantitative risk--benefit methodologies for assessing drug safety and efficacy—report of the ispor risk--benefit management working group. Value in Health\/ 13\/ (5), 657--666

    work page 2010

  21. [21]

    Holden, W. L. (2003). Benefit-risk analysis. Drug safety\/ 26\/ (12), 853--862

    work page 2003

  22. [22]

    Jong, N. K. and P. Stone (1976). Keeney, rl &raiffa, h. decisions with multiple objectives: Preferences and value tradeoffs. In In Proceedings of the ICML-06 Workshop on Kernel Methods in Reinforcement Learning . Citeseer

    work page 1976

  23. [23]

    Keeney, R. L., H. Raiffa, and R. F. Meyer (1993). Decisions with multiple objectives: preferences and value trade-offs . Cambridge university press

    work page 1993

  24. [24]

    Lerch, T

    Kr \"u ger, F., S. Lerch, T. Thorarinsdottir, and T. Gneiting (2021). Predictive inference based on markov chain monte carlo output. International Statistical Review\/ 89\/ (2), 274--301

    work page 2021

  25. [25]

    Hokkanen, and P

    Lahdelma, R., J. Hokkanen, and P. Salminen (1998). Smaa-stochastic multiobjective acceptability analysis. European Journal of Operational Research\/ 106\/ (1), 137--143

    work page 1998

  26. [26]

    Kurowicka, and H

    Lewandowski, D., D. Kurowicka, and H. Joe (2009). Generating random correlation matrices based on vines and extended onion method. Journal of Multivariate Analysis\/ 100\/ (9), 1989 -- 2001

    work page 2009

  27. [27]

    Li, K., S. Luo, S. Yuan, and S. Mt-Isa (2019). A bayesian approach for individual-level drug benefit-risk assessment. Statistics in medicine\/ 38\/ (16), 3040--3052

    work page 2019

  28. [28]

    Lynd, L. D. and B. J. O'brien (2004). Advances in risk-benefit evaluation using probabilistic simulation methods: an application to the prophylaxis of deep vein thrombosis. Journal of clinical epidemiology\/ 57\/ (8), 795--803

    work page 2004

  29. [29]

    Meng, X.-L. (1994). Posterior predictive p -values. The Annals of Statistics\/ 22\/ (3), 1142--1160

    work page 1994

  30. [30]

    Moustaki, I. (1996). A latent trait and a latent class model for mixed observed variables. British journal of mathematical and statistical psychology\/ 49\/ (2), 313--334

    work page 1996

  31. [31]

    Mt-Isa, S., C. E. Hallgreen, N. Wang, T. Callr \'e us, G. Genov, I. Hirsch, S. F. Hobbiger, K. S. Hockley, D. Luciani, L. D. Phillips, et al. (2014). Balancing benefit and risk of medicines: a systematic review and classification of available methodologies. Pharmacoepidemiology and drug safety\/ 23\/ (7), 667--678

    work page 2014

  32. [32]

    M \"u hlbacher, A. C., C. Juhnke, A. R. Beyer, and S. Garner (2016). Patient-focused benefit-risk analysis to inform regulatory decisions: the european union perspective. Value in Health\/ 19\/ (6), 734--740

    work page 2016

  33. [33]

    Salek, and S

    Mussen, F., S. Salek, and S. Walker (2007). A quantitative approach to benefit-risk assessment of medicines--part 1: the development of a new model using multi-criteria decision analysis. Pharmacoepidemiology and drug safety\/ 16\/ (S1), S2--S15

    work page 2007

  34. [34]

    Salek, and S

    Mussen, F., S. Salek, and S. Walker (2009). Benefit-risk appraisal of medicines. A systematic approach to decision-making\/

    work page 2009

  35. [35]

    Muth\' e n, B. and T. Asparouhov (2012). Bayesian S tructural E quation M odeling: A more flexible representation of substantive theory. Psychological Methods\/ 17 , 313--335

    work page 2012

  36. [36]

    Neal, R. M. (2011). Mcmc using hamiltonian dynamics. Handbook of markov chain monte carlo\/ 2\/ (11), 2

    work page 2011

  37. [37]

    Nissen, S. E. and K. Wolski (2007). Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. New England Journal of Medicine\/ 356\/ (24), 2457--2471

    work page 2007

  38. [38]

    Ponce, R. A., S. M. Bartell, E. Y. Wong, D. LaFlamme, C. Carrington, R. C. Lee, D. L. Patrick, E. M. Faustman, and M. Bolger (2000). Use of quality-adjusted life year weights with dose-response models for public health decisions: A case study of the risks and benefits of fish consumption. Risk Analysis\/ 20\/ (4), 529--542

    work page 2000

  39. [39]

    Hoijtink, and A

    Scheines, R., H. Hoijtink, and A. Boomsma (1999). Bayesian estimation and testing of structural equation models. Psychometrika\/ 64\/ (1), 37--52

    work page 1999

  40. [40]

    Shaffer, M. L. and K. L. Watterberg (2006). Joint distribution approaches to simultaneously quantifying benefit and risk. BMC medical research methodology\/ 6\/ (1), 1--8

    work page 2006

  41. [41]

    Shakespeare, T. P., V. J. Gebski, M. J. Veness, and J. Simes (2001). Improving interpretation of clinical studies by use of confidence levels, clinical significance curves, and risk-benefit contours. The Lancet\/ 357\/ (9265), 1349--1353

    work page 2001

  42. [42]

    Sinharay, S. (2005). Assessing fit of unidimensional item response theory models using a Bayesian approach. Journal of Educational Measurement\/ 42\/ (4), 375--394

    work page 2005

  43. [43]

    Sutton, A. J., N. J. Cooper, K. R. Abrams, P. C. Lambert, and D. R. Jones (2005). A bayesian approach to evaluating net clinical benefit allowed for parameter uncertainty. Journal of clinical epidemiology\/ 58\/ (1), 26--40

    work page 2005

  44. [44]

    Tervonen, T. (2014). Jsmaa: open source software for smaa computations. International Journal of Systems Science\/ 45\/ (1), 69--81

    work page 2014

  45. [45]

    Tervonen, T. and J. R. Figueira (2008). A survey on stochastic multicriteria acceptability analysis methods. Journal of Multi-Criteria Decision Analysis\/ 15\/ (1-2), 1--14

    work page 2008

  46. [46]

    Van Valkenhoef, E

    Tervonen, T., G. Van Valkenhoef, E. Buskens, H. L. Hillege, and D. Postmus (2011). A stochastic multicriteria model for evidence-based decision making in drug benefit-risk analysis. Statistics in Medicine\/ 30\/ (12), 1419--1428

    work page 2011

  47. [47]

    Kalogeropoulos, and I

    Vamvourellis, K., K. Kalogeropoulos, and I. Moustaki (2021). Generalised bayesian structural equation modelling. arXiv preprint arXiv:2104.01603\/

    work page arXiv 2021

  48. [48]

    Kalogeropoulos, and I

    Vamvourellis, K., K. Kalogeropoulos, and I. Moustaki (2022). Sequential bayesian inference for factor analysis

    work page 2022

  49. [49]

    Ntzoufras, and I

    Vitoratou, V., I. Ntzoufras, and I. Moustaki (2014). Marginal likelihood estimation from the metropolis output: tips and tricks for efficient implementation in generalized linear latent variable models. Journal of Statistical Computation and Simulation\/ 84\/ (10), 2091--2105

    work page 2014

  50. [50]

    Mt-Isa, R

    Waddingham, E., S. Mt-Isa, R. Nixon, and D. Ashby (2016). A bayesian approach to probabilistic sensitivity analysis in structured benefit-risk assessment. Biometrical Journal\/ 58\/ (1), 28--42

    work page 2016

  51. [51]

    Wallach, J. D., K. Wang, A. D. Zhang, D. Cheng, H. K. G. Nardini, H. Lin, M. B. Bracken, M. Desai, H. M. Krumholz, and J. S. Ross (2020). Updating insights into rosiglitazone and cardiovascular risk through shared data: individual patient and summary level meta-analyses. bmj\/ 368

    work page 2020