{"id":"7d8260a8-de30-449d-ac7b-6a42526cc61a","arxiv_id":"2505.04727","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"An EM algorithm for proportional odds ordinal regression with outcome-dependent nonignorable missing responses is proposed, simulated, and applied to a Phase III psoriasis trial.","lead":"This paper introduces an EM algorithm for fitting proportional odds regression models when ordinal outcomes are missing not at random, meaning the chance of missingness depends on the unobserved outcome itself. It is a practical sensitivity tool for clinical trials where dropout can bias complete-case treatment estimates.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Simulation tables contradict EM's claimed consistent superiority over CC and MI at 25% and 45% missingness.","rationale":"The reader identified identifiability and global convergence as the weakest assumption, but the more direct and decisive concern is that the paper's own simulation results disprove the central claim of consistent superiority. At 25% and 45% missingness, EM is often the worst performer, including at large sample sizes, which rules out a small-sample explanation. This is an internal inconsistency between the text and the reported tables, not just a missing proof. It undermines the paper's main contribution. The verdict should move from CONDITIONAL to REJECT because the headline empirical claim is falsified by the presented evidence. The paper could potentially be revised by drastically narrowing the claims and adding consistency diagnostics, but in its current form the central argument does not hold.","tokens_in":25216,"tokens_out":7666,"duration_ms":74893,"concrete_test":"From Tables 3 and 4, count for each of the 5 parameters across all sample sizes how often EM has lower absolute bias and lower MSE than both CC and MI. The central claim requires EM to win for a clear majority of rows; if EM wins in fewer than half the rows (or none at n=1000), the claim is contradicted by the paper's own evidence. As a consistency check, re-run Simulation 3 at n=5000 with the exact Table 1 parameters; if EM's absolute bias for y≤1, y≤2, and x1 remains above CC's bias or does not decrease by an order of magnitude from n=1000, the estimator is not consistent in the setting used to support the claim.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's central claim, stated in Section 4.2, is that the proposed EM algorithm 'consistently outperforms CC and MI in terms of reducing bias and MSE.' The paper's own simulation tables contradict this. In Table 4 (45% missing, n=1000), EM absolute biases are 2.989 (y≤1), 1.682 (y≤2), 0.850 (x1), 0.024 (x3), and 0.036 (x4), while CC biases are 0.000, 0.008, 0.001, 0.000, and 0.000, and MI biases are 1.465, 1.299, 0.214, 0.008, and 0.042. EM is the worst method for every parameter except x4, where it narrowly beats MI. Table 3 (25% missing, n=1000) shows the same pattern: EM has the largest bias for all five parameters. The bias does not shrink between n=500 and n=1000, so this is not a small-sample artifact or a slow convergence rate; it suggests the EM estimator may be inconsistent even under the correctly specified data-generating mechanism. Section 3.5's convergence discussion, based on Fisher consistency of surrogate losses, does not address the incomplete-data likelihood or the behavior of the EM fixed point. Since the contribution rests on empirical superiority, this internal contradiction is load-bearing: if EM is not consistently better, the central claim fails.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an EM algorithm for fitting a cumulative logit proportional odds ordinal regression model when the response variable is missing not at random (MNAR). The missingness mechanism is modeled by a logistic regression that includes the outcome itself as a covariate (equations 3 and 8), and the E-step weights each possible value of the missing ordinal response by its posterior probability under the current parameter estimates. The M-step is performed by fitting a weighted proportional odds model and a weighted logistic model (Section 3.2). The authors evaluate the method in simulations with sample sizes 60 to 1000 and missingness rates 10%, 25%, and 45%, comparing complete-case analysis (CC), multiple imputation (MI) via MICE, and the proposed EM approach, and they apply the method to a Phase III psoriasis trial with a five-category PGA endpoint. The central claim in Section 4.2 is that the EM algorithm consistently outperforms CC and MI in reducing bias and MSE.","tokens_in":25508,"tokens_out":6261,"duration_ms":58827,"significance":"If the method delivered the claimed improvements, it would be a practically useful extension of Ibrahim and Lipsitz (1996) to ordinal outcomes in clinical trials, where nonignorable missingness is a common concern. The paper provides a concrete algorithmic recipe, an extensive simulation grid, a real-data application, and a SAS macro, which are valuable for reproducibility. However, the manuscript's own tables contradict the headline claim of consistent superiority at 25% and 45% missingness, and the theoretical convergence argument in Section 3.5 is not valid as stated. The method appears to perform well at 10% missingness, so the contribution is potentially salvageable, but the current claims and supporting evidence need substantial revision.","major_comments":[{"comment":"The central claim that EM 'consistently outperforms CC and MI in terms of reducing bias and MSE' is contradicted by the paper's own simulation results at higher missingness. At n=1000 with 25% missing (Table 3), EM absolute biases for y≤1, y≤2, x1, x3, and x4 are 1.609, 0.782, 0.554, 0.016, and 0.027, whereas CC biases are 0.001, 0.007, 0.001, 0.000, and 0.000, and MI biases are 0.963, 0.693, 0.020, 0.009, and 0.017. At 45% missing (Table 4, n=1000), EM biases are 2.989, 1.682, 0.850, 0.024, and 0.036, compared with CC biases of 0.000, 0.008, 0.001, 0.000, and 0.000 and MI biases of 1.465, 1.299, 0.214, 0.008, and 0.042. EM is the worst method for nearly every parameter in these settings, and the bias does not shrink from n=500 to n=1000, which is inconsistent with the claimed consistency. The authors should either correct the results, explain why EM behaves so poorly at higher missingness, or substantially revise the claim of consistent superiority.","section":"§4.2, Tables 3 and 4"},{"comment":"The convergence discussion is not a proof. The paper states that Fisher consistency of the cumulative logit surrogate loss (Pedregosa et al., Theorem 7) provides an 'indirect justification for the existence of a global maximum for Q(·|·)' and that minimizing a surrogate loss is equivalent to maximizing a likelihood. Fisher consistency is a property of the population risk minimizer for classification and does not imply unimodality of a finite-sample incomplete-data Q function, nor does it address the behavior of the EM fixed point. The authors should either provide a valid convergence argument (for example, standard EM ascent with explicit conditions on the observed-data likelihood) or explicitly state that only convergence to a stationary point is guaranteed.","section":"§3.5"},{"comment":"Identifiability of the joint model (β, α) is never discussed. With Y missing not at random and p(R|X,Y,α) depending on the unobserved outcome, the observed-data likelihood can be non-identifiable without additional restrictions. Since the consistency claim in Section 4.2 rests on the likelihood being well-behaved, the paper should provide conditions under which the parameters are identified from the observed data, or at least acknowledge the issue and report a sensitivity analysis. This is load-bearing for the theoretical validity of the estimator.","section":"Sections 2 and 3"},{"comment":"The simulation summaries are computed 'after excluding non-converged runs,' but the number and proportion of excluded runs are never reported. If EM fails to converge in a sizable fraction of replications, the reported means, MSEs, and coverage probabilities are conditional on convergence and may be systematically misleading. The authors should report convergence rates for every scenario, especially at n=60 and 45% missingness where the EM estimates show very large biases.","section":"§4.2"},{"comment":"The supplementary five-category simulation also undercuts the paper's stated superiority. The text says EM is the second best method after 'whole' and outperforms CC, but Table 1 at n=1000 shows EM absolute bias of 0.923 for y≤1 compared with MI's 0.522 and CC's 0.000, and EM has the largest bias among the three for all five parameters. The discrepancy between the claims and the reported numbers needs to be addressed.","section":"Supplementary Section 2.1 and Table 1"}],"minor_comments":[{"comment":"The keyword 'Nonignorbale' is a typo and should be 'Nonignorable'.","section":"Keywords"},{"comment":"In the sentence 'In the above model (7, the variables...' the opening parenthesis after 'model' is missing its closing parenthesis; and 'condfidence' should be 'confidence'.","section":"§5"},{"comment":"Equation (7) uses j = 0, 1, 2, 3 for a five-category outcome, while the general model in equation (1) uses j = 1, ..., J-1. The indexing should be harmonized to avoid confusion.","section":"Equation (7)"},{"comment":"The entry for the MI estimate in the x4 row appears misaligned or erroneous (the value 0.102 would imply a bias of 0.202 against the true -0.100, yet the reported absolute bias is 0.001). Please check the alignment of this row.","section":"Table 2, n=500, x4 row"},{"comment":"In the supplementary figures, the top-level labels 'β2 β3' are ambiguous because the panels display β01, β02, β1, β2, and β3. Clarify the panel labels and the mapping from β2/β3 to x3/x4 in Tables 2-4.","section":"Figure labels and table headers"},{"comment":"The citation 'Firth ( ?)' is unresolved; please provide the full reference for the Firth correction.","section":"Supplementary Table 4"},{"comment":"The statement that the 95% CP of EM approaches 0.95 as n increases is not supported by Table 4 at n=1000, where CP for y≤1 is 0.932 and for x3 is 0.960. Please revise the summary to be consistent with the tables, and soften Section 6's claim that the convergence properties are 'supported by theoretical guarantees.'","section":"§4.2 and §6"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's headline claim of consistent superiority over CC and MI is directly contradicted by its own Tables 3 and 4, so the paper cannot be accepted as is. The situation may be salvageable if the authors re-examine the EM implementation, report convergence failures, and reframe the contribution as an EM algorithm that works well at low missingness but has identifiability/convergence limitations that need careful discussion. The theoretical section (3.5) should be rewritten to avoid invoking Fisher consistency of surrogate losses as a proof of global convergence. I would also encourage the editor to ask for the simulation code so that the discrepancies between the text and tables can be checked."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about arXiv:2505.04727. First, the problem it tackles is real: ordinal trial endpoints with dropout that depends on the unmeasured outcome are a running worry in clinical stats, and a nonignorable-missingness EM for cumulative logit proportional odds models is a sensible gap to fill. Second, the paper's own simulation tables undermine its central claim. In the 25% and 45% missingness scenarios, the proposed EM does not consistently beat complete-case or MICE; for several parameters, especially the intercepts, EM has larger bias than either competitor, and the bias does not shrink with sample size. That is not a minor caveat—it is the empirical foundation of the paper.\n\nWhat is genuinely new is the mechanics: the weighted data-augmentation scheme of Ibrahim and Lipsitz, extended from binary to J-category cumulative logit, with standard errors via Louis's method. The derivation is standard but legitimate, and the SAS program is a concrete reproducible artifact. The psoriasis application adds a real-data check, and the significant coefficient on Y in the missingness model gives the MNAR story some bite. If the simulations were trustworthy, this would be a useful sensitivity-analysis tool for trialists.\n\nThey are not trustworthy as reported. Section 3.5 calls itself a convergence proof but actually leans on Fisher consistency of ordinal surrogate losses, which concerns classification risk, not the shape of the observed-data likelihood. Identifiability is never discussed, and non-converged runs are excluded without counts. More telling, in the 45% missingness table the complete-case estimates are nearly identical to the full-data estimates, which is implausible under strong outcome-dependent missingness. Either the simulation code is wrong or the table is mislabeled. The EM biases are large enough to suggest a coding bug or serious numerical issues, not just a slow rate.\n\nBottom line: the extension is fine, but the evidence is internally contradictory. The numbers need to be reconciled—and possibly the implementation fixed—before the method can be taken as reliable. I would still send this to a statistician who can re-run the simulations, because if the error is findable, the paper may be publishable after major revision. But I would not cite it as evidence until then.\n\nBest.","headline":"The ordinal EM extension is a sensible piece of work, but the paper's own simulation tables contradict its headline claim, and the complete-case results look implausible, so the empirical core needs a hard recheck before the method can be trusted.","tokens_in":25996,"tokens_out":6111,"would_cite":false,"duration_ms":61530,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F10","62J12","62P10"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims an EM algorithm that jointly fits a cumulative-logit proportional-odds model and a logistic missingness model containing the outcome reduces bias and mean squared error, relative to complete-case analysis and MICE-based…","keywords":["cumulative logit","EM algorithm","nonignorable missing","ordinal response","proportional odds","randomized trial","missing not at random"],"falsifier":"Replicate the $n=60$, 45%-missingness scenario of the paper's Table 4 and compare EM with MICE on the treatment coefficient; the paper's own table shows MICE with bias 0.102 versus EM's 0.583, so this cell already questions the claim of uniform superiority, and repeating the protocol under a threshold or quadratic missingness mechanism would test robustness to model misspecification.","tokens_in":25028,"feed_emoji":"📊","tokens_out":8810,"duration_ms":83075,"temperature":0.7,"pith_summary":"Clinical trials with ordinal endpoints lose information when outcomes are missing, and the bias is worst when missingness depends on the outcome itself, a nonignorable or missing-not-at-random mechanism. This paper proposes an expectation-maximization (EM) algorithm for cumulative-logit proportional-odds regression that jointly models the ordinal response and a logistic missingness model in which the outcome appears as a predictor. In simulations with about 10%, 25%, and 45% missing responses and sample sizes from 60 to 1000, the EM estimates show lower bias and mean squared error than complete-case analysis and MICE-based multiple imputation, with coverage probabilities approaching 0.95 as the sample grows. Applied to a Phase III psoriasis trial where 13.5% of five-point Physician's Global Assessment scores were missing, the method estimates a significant outcome-missingness association and yields treatment odds ratios that differ from both complete-case and MICE estimates.","feed_headline":"EM cuts bias for missing ordinal outcomes in trials","feed_subtitle":"Modeling dropout with the outcome itself beats complete-case analysis and MICE in a Phase III psoriasis trial.","key_machinery":"The carrying object is the joint likelihood $L(\\boldsymbol{\\gamma}) = \\prod_i f_i(Y_i \\mid X_i, \\boldsymbol{\\beta}) \\, p_i(R_i \\mid Z_i, \\boldsymbol{\\alpha})$, where $Z_i=(X_i, Y_i)$ and $p_i$ is a logistic missingness probability. The E-step assigns a missing response the weight $w_{iy} = f_i(Y_i\\mid X_i,\\boldsymbol{\\beta}) p_i(R_i\\mid Z_i,\\boldsymbol{\\alpha}) / \\sum_{j=1}^J f_i(Y_i=j \\mid X_i,\\boldsymbol{\\beta}) p_i(R_i\\mid Z_i,\\boldsymbol{\\alpha})$, turning each missing observation into $J$ weighted pseudo-observations, one per ordinal category. The M-step then maximizes the weighted proportional-odds likelihood and the weighted logistic missingness likelihood separately, using standard software; iteration continues to convergence, and the final covariance matrix uses the Louis correction for the observed information. Because the missingness model includes $Y$ itself, the weights are the conditional predictive distribution of the missing outcome given covariates and missingness, which is the mechanism that removes the nonignorable bias.","core_discovery":"The central claim is that explicit modeling of the missingness mechanism removes most of the bias that complete-case analysis and MAR-based multiple imputation leave in proportional-odds ordinal regression when responses are missing not at random. The paper builds the joint likelihood as the product of a cumulative-logit model for the ordinal response $Y$ and a binary logistic model for the missing indicator $R$, with the unobserved $Y$ included as a covariate in the missingness model, so a nonzero coefficient on $Y$ signals nonignorable missingness. An EM algorithm treats each missing response as a weighted mixture over all ordinal categories, re-estimates the weights from the fitted models, and iterates between weighted proportional-odds and weighted logistic fits. Over 1000-replication simulations the EM estimator is reported to dominate complete-case and MICE in bias and mean squared error across missingness rates, with near-nominal coverage at larger sample sizes, and standard errors are computed from the Louis observed-information correction. In the psoriasis application the outcome coefficient in the missingness model is significant, and the EM-based odds ratios for both active doses are attenuated relative to complete-case estimates, while onset age becomes significant only under EM.","pith_inferences":["If the true missingness mechanism is nonlinear in the outcome, such as dropout driven by an extreme-category threshold, the linear logistic model for $R$ is misspecified and could restore bias; testing the method under such misspecification would delimit its scope.","The same reweighting principle should transfer to other ordinal links such as probit or complementary log-log, and to ordinal mixed models where nonignorable dropout is common in longitudinal trials.","At the smallest sample size with heavy missingness, the paper's own tables show EM's intercept estimates still carry large absolute bias, so the method is best presented as a moderate-to-large-sample procedure rather than a universal fix.","A practical extension is to embed the outcome-coefficient test in routine sensitivity analyses: if the coefficient on $Y$ is insignificant, analysts could reasonably report the simpler MAR-based results."],"forward_implications":["Treatment-effect estimates in ordinal clinical-trial endpoints will move closer to the complete-data values when missingness depends on the outcome, making the method a concrete sensitivity check for regulatory submissions.","The coefficient on the outcome in the missingness model can be tested, giving a diagnostic for whether missingness is likely nonignorable.","Confidence intervals built from the Louis information matrix approach nominal coverage at moderate-to-large sample sizes, supporting valid inference rather than only point estimation.","Because the E-step re-weights all possible response categories, no ad hoc imputation model or rounding rule for ordinal values is needed.","The released SAS macro makes the procedure usable in the same software environment many clinical-trial statisticians already employ."],"supporting_citations":[{"why":"Supplies the EM-with-nonignorable-missing-responses framework that the paper extends from binomial regression to ordinal cumulative-logit regression.","marker":"[9]"},{"why":"Earlier extension of that EM framework to logistic regression, cited as evidence the approach performs well before adaptation to ordinal outcomes.","marker":"[10]"},{"why":"Provides the derivative structure for cumulative link models used in the M-step score equations.","marker":"[11]"},{"why":"Gives the observed-information formula used to compute standard errors and confidence intervals after EM convergence.","marker":"[12]"},{"why":"Fisher consistency of cumulative-logit surrogate losses is cited in Section 3.5 to argue the weighted likelihood has a unique global maximum.","marker":"[18]"},{"why":"Defines the MCAR/MAR/MNAR taxonomy and the notion of ignorable missingness that the proposed method targets.","marker":"[2]"},{"why":"The chained-equations multiple imputation implementation used as the main comparator in simulations.","marker":"[19]"},{"why":"The Phase III psoriasis trial data provide the motivating application and the reference for treatment-effect comparisons.","marker":"[1]"}],"fun_headline_variants":["EM fixes bias when missingness depends on the outcome","Proportional odds with missing-not-at-random: EM wins","Model missingness with the outcome itself to cut bias","EM for ordinal outcomes under missing-not-at-random","Nonignorable missingness? EM adjusts ordinal odds better"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method works only if the assumed logistic missingness model, with the outcome entering as a linear predictor, is the true mechanism and the EM likelihood has a single peak; the paper does not prove the single peak, and a wrong missingness model could produce estimates as biased as the simpler approaches.","fun_headline_variants_meta":{"raw":{"variants":["EM fixes bias when missingness depends on the outcome","Proportional odds with missing-not-at-random: EM wins","Model missingness with the outcome itself to cut bias","EM for ordinal outcomes under missing-not-at-random","Nonignorable missingness? EM adjusts ordinal odds better"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00073,"raw_usage":{"total_tokens":3260,"prompt_tokens":927,"completion_tokens":2333,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":2256}},"tokens_in":543,"tokens_out":2333,"duration_ms":16948,"temperature":1.0,"reasoning_tokens":2256,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:22:32.311475+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replicate the $n=60$, 45%-missingness scenario of the paper's Table 4 and compare EM with MICE on the treatment coefficient; the paper's own table shows MICE with bias 0.102 versus EM's 0.583, so this cell already questions the claim of uniform superiority, and repeating the protocol under a threshold or quadratic missingness mechanism would test robustness to model misspecification.","supporting_citations":[{"cited_title":"Parameter estimation from incomplete data in binomial regression when the missing data mechanism is nonignorable.Biometrics1996: 1071–1078","cited_arxiv_id":null,"evidence_quote":"Supplies the EM-with-nonignorable-missing-responses framework that the paper extends from binomial regression to ordinal cumulative-logit regression."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Earlier extension of that EM framework to logistic regression, cited as evidence the approach performs well before adaptation to ordinal outcomes."},{"cited_title":"Improved estimation in cumulative link models.Journal of the Royal Statistical Society Series B: Statistical Methodology2014; 76(1): 169–196","cited_arxiv_id":null,"evidence_quote":"Provides the derivative structure for cumulative link models used in the M-step score equations."},{"cited_title":"Journal of the Royal Statistical Society: Series B (Statisticial Methodology) 1982; 44(2): 226–233","cited_arxiv_id":null,"evidence_quote":"Gives the observed-information formula used to compute standard errors and confidence intervals after EM convergence."},{"cited_title":"Journal of Machine Learning Research 2017; 18: 1–35","cited_arxiv_id":null,"evidence_quote":"Fisher consistency of cumulative-logit surrogate losses is cited in Section 3.5 to argue the weighted likelihood has a unique global maximum."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the MCAR/MAR/MNAR taxonomy and the notion of ignorable missingness that the proposed method targets."},{"cited_title":"mice: Multivariate imputation by chained equations in R.Journal of Statistical Software2011; 45: 1–67","cited_arxiv_id":null,"evidence_quote":"The chained-equations multiple imputation implementation used as the main comparator in simulations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The Phase III psoriasis trial data provide the motivating application and the reference for treatment-effect comparisons."}],"review_version":1}