{"id":"bf231433-a43b-430b-8383-9a1094779624","arxiv_id":"1908.05338","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A robust parametric model that jointly fits modified logistic curves to Alzheimer's biomarkers on an aligned disease timeline achieves the best reported prediction accuracy on ADNI and generalizes to the NACC cohort.","lead":"This paper introduces a robust statistical model that tracks Alzheimer's disease progression by aligning patients' ages on a single disease timeline and fitting curves to several biomarkers at once. It reports better predictions of biomarker values and clinical status than previous methods, and shows the model trained on one large dataset generalizes to another.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Test NMAE/AUC are fit residuals from each test subject's full data, so the claimed state-of-the-art comparison is not apples-to-apples.","rationale":"The reader's weakest assumption was the single-latent-DPS linear mapping, which is a modeling validity concern. I agree it matters for the scientific interpretation of inflection-point ordering, but it does not directly threaten the reported NMAE/AUC numbers: a misspecified model can still fit and classify well. The Stannard/Richards equivalence is a real novelty problem but again does not change the empirical ranking. The most load-bearing issue for the central claim is the evaluation protocol underlying the headline 'state-of-the-art' comparison. If test DPSs are estimated from the full longitudinal record, the reported MAE and AUC are in-sample fit statistics and cannot be compared to forecasting results from TADPOLE. This directly affects the claim of outperforming almost all state-of-the-art results. The proposed concrete check settles this by rerunning under the official forecast protocol. If the adjusted MAE/AUC match the reported numbers, my concern is unfounded; if they worsen materially, the paper's central comparative claim must be downgraded.","tokens_in":19210,"tokens_out":12342,"duration_ms":109094,"concrete_test":"Re-run the TADPOLE D1/D2 evaluation under the official forecast protocol: for each D2 subject, estimate α_i and β_i using only biomarker measurements up to the TADPOLE forecast date, then predict ADAS-13 and ventricle volume at 12, 24, and 36 months and compute the multiclass AUC from the predicted DPSs. If the ADAS-13 MAE exceeds 4.70 or the AUC falls below 0.931, the claimed superiority over TADPOLE state-of-the-art is refuted.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The paper's test-set numbers are not forecasting errors. In Sec. 2.3, for each test subject the subject-specific DPS parameters (α_i, β_i) are estimated by minimizing Eq. (2) over all available biomarker values and visits of that subject. The reported MAE is therefore the residual of a per-subject curve fit after the test subject's own later measurements have been used to align the trajectory. This is a legitimate measure of trajectory-shape generalization, and the comparison to the analogous Jedynak model (Table 6) is fair because both methods use the same protocol. The problem is the claim in Sec. 4.6 and the abstract that the model 'outperforms almost all state-of-the-art results in predicting biomarker values and classifying clinical status.' The TADPOLE challenge results cited (MAE 4.70 for ADAS-13, AUC 0.931) are true forecasting metrics: predictions are made from a fixed forecast date using only data up to that date. The proposed method's 3.93 ADAS-13 MAE and 0.937 AUC appear to be computed with test DPSs estimated from the full longitudinal record, including the target future visits, which leaks information about the very values being 'predicted'. Unless the TADPOLE evaluation is run under the official forecast protocol, the state-of-the-art comparison is invalid and the strongest claim in the abstract is unsupported.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a robust parametric disease progression model for Alzheimer's disease, extending the approach of Jedynak et al. (2012, 2015). The model maps each subject's age to a disease progression score (DPS) via a subject-specific linear transformation and jointly fits constrained generalized logistic curves to longitudinal biomarker data using M-estimation. The authors evaluate several logistic functions and loss functions, introduce what they call the modified Stannard function, use bootstrapping to quantify uncertainty, order biomarkers by estimated inflection points, and classify clinical status using kernel density estimates of DPS distributions with a Bayesian classifier. On ADNI test data, the best configuration (modified Stannard with logistic loss) is reported to achieve an average normalized MAE of 0.991 and a multiclass AUC of 0.934; on NACC data the corresponding values are 1.182 and 0.929. The paper also claims to outperform most state-of-the-art results in predicting biomarker values and classifying clinical status.","tokens_in":19469,"tokens_out":4980,"duration_ms":52110,"significance":"If the results are validated under an appropriate protocol, the work would make a useful contribution: robust M-estimation for disease progression modeling, joint fitting of multiple biomarkers, bootstrap-based uncertainty quantification, and cross-cohort evaluation are all valuable components, and the comparison with the analogous Jedynak model under the same protocol is a legitimate like-for-like assessment. The temporal ordering of biomarkers is consistent with prior literature, which lends credibility to the fitted curves. However, two load-bearing issues compromise the current claims: the modified Stannard function is mathematically a reparameterization of Richards' function, so the claimed novelty is inaccurate; and the comparison with TADPOLE and other state-of-the-art methods is not apples-to-apples because the test DPSs are estimated from each test subject's full longitudinal record, including the visits whose values are later 'predicted.' The paper's central contribution is therefore better described as a robust, jointly fitted extension of an existing parametric model rather than a new functional form with state-of-the-art forecasting performance.","major_comments":[{"comment":"The claimed state-of-the-art comparison in Section 4.6 is not supported by the evaluation protocol. In Section 2.3, for each test subject the subject-specific DPS parameters (alpha_i, beta_i) are estimated by minimizing Eq. (2) over all available biomarker values and visits for that subject, including the same visits whose biomarker values are subsequently predicted. The TADPOLE official evaluation, by contrast, requires predictions to be made from a fixed forecast date using only data available up to that date. The reported ADAS-13 MAE of 3.93 and AUC of 0.937 are therefore not comparable to the cited TADPOLE results (MAE 4.70, AUC 0.931). The authors should either rerun the evaluation under the official TADPOLE forecast protocol or remove the state-of-the-art comparison and revise the abstract and conclusion accordingly.","section":"Section 4.6 and Section 2.3"},{"comment":"The 'modified Stannard' function is not a new family of growth functions. Its formula, g(s) = [1 + (1/gamma) exp(-(b/gamma)(s-c))]^{-gamma}, becomes the Richards function after the reparameterization r = 1/gamma and b_r = b/gamma: [1 + r exp(-b_r(s-c))]^{-1/r}. Consequently, the claimed four-fold contribution of introducing a novel logistic function is incorrect, and the small BIC differences between Richards and modified Stannard in Table 5 are numerical artifacts of the optimization rather than evidence of a genuinely more flexible model. The novelty claim should be removed or the contribution reframed around the robust estimation and joint fitting components.","section":"Table 1 and Section 1"},{"comment":"The test-set NMAE and AUC are not forecasting metrics in the usual sense. Because the test DPSs are estimated from the complete longitudinal record of each test subject, the reported errors measure the quality of a per-subject trajectory fit, not the accuracy of predictions made from an earlier time point. The comparison with the Jedynak model in Table 6 is fair as a like-for-like trajectory-shape comparison, and this is a legitimate evaluation target, but the manuscript should consistently describe the results as trajectory-modeling and trajectory-based classification performance, not as state-of-the-art biomarker prediction. This distinction is essential for readers comparing the results with TADPOLE and other longitudinal forecasting benchmarks.","section":"Sections 4.4 and 4.5"}],"minor_comments":[{"comment":"The definition of BIC uses E(Lopt)_train, while Sections 3.3 and 3.4 describe selecting Lopt using the validation loss; please clarify whether the reported BIC is computed from the training loss at the validation-selected iteration and how this avoids optimistic model selection.","section":"Section 3.2"},{"comment":"The 16 by 16 probability matrix in Figure 5 is very dense and difficult to read at print size; enlarging the figure or presenting the ordering probabilities in a supplementary table would improve accessibility.","section":"Figure 5"},{"comment":"The columns 'Inflection point (median)' and 'Test MAE (mean)' in Table 7 should state their units and clarify that the MAE values are averaged over the 100 bootstrap runs.","section":"Table 7"},{"comment":"The 3-month matching rule for MRI and clinical visits is reasonable, but the manuscript should report how many MRI visits in NACC were assigned missing clinical status as a consequence, since this affects the effective sample size for the classification experiments.","section":"Section 3.1.3"},{"comment":"The statement that the method could be applied to biomarkers with non-monotonic behavior 'as long as suitable functions are used' is vague; a concrete example or a reference to a suitable function family would make the claim more informative.","section":"Section 5"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid experimental framework in terms of bootstrapping, data-split release, and a fair head-to-head comparison with the Jedynak baseline under identical conditions. The two main problems are the inaccurate novelty claim for the modified Stannard function and the invalid state-of-the-art comparison due to information leakage in the test protocol. Both are fixable within the manuscript's scope by reanalysis or by substantially revising the claims, so rejection is not warranted. I would ask the authors to either run the TADPOLE evaluation under the official forecast protocol or clearly separate their trajectory-fitting evaluation from true forecasting benchmarks."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the paper's claimed novel curve, 'modified Stannard,' is a reparameterization of the Richards function already in the same table: set gamma' = 1/gamma and b' = b/gamma and the two families coincide. So the headline novelty is not new. Second, the test-set numbers are not forecasting errors. The test subject's own full longitudinal record is used to estimate that subject's DPS parameters, and then the same values are 'predicted.' That makes the comparison to TADPOLE (which uses a fixed forecast date) unfair, and the abstract's claim to outperform state-of-the-art is unsupported.\n\nWhat's actually good: the joint M-estimation fitting is a sensible extension of Jedynak's alternating scheme, the bootstrapped evaluation is solid, the head-to-head against the Jedynak baseline in Table 6 is apples-to-apples (both methods use the same test-subject DPS estimation), and the cross-cohort transfer to NACC is genuinely useful. The data splits are released as ancillary files, which is real reproducibility support even though no code is shipped.\n\nThe soft spots: besides the two above, the data filtering in Section 3.1.3 removes visits with reverting diagnoses and sets a three-month time-gap threshold 'based on the data statistics,' which is a post hoc choice that can bias the results. The KDE bandwidth w is never reported. The inflection-point ordering is consistent with prior work, so it's not a new biological finding.\n\nThis paper is for people working on parametric disease progression models, especially those comparing DPMs on ADNI/NACC. It deserves a serious referee because the core method and the within-paper comparisons are reproducible and the work is honest in its handling of the literature. But the revision should be major: disclose the Richards equivalence, rerun the TADPOLE comparison under the official forecast protocol, report sensitivity of the filtering choices, and release the code.","headline":"Solid engineering contribution undermined by a false novelty claim (modified Stannard is just reparameterized Richards) and a test protocol that turns the state-of-the-art comparison into a self-fit exercise.","tokens_in":20042,"tokens_out":2464,"would_cite":true,"duration_ms":25063,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62P10","62F35","62G07","62H30"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a single latent disease progression score, linearly mapped from age and shared across biomarkers, suffices to model Alzheimer's progression when biomarker curves are fit jointly with a robust M-estimator and a…","keywords":["Alzheimer's disease","disease progression modeling","M-estimation","generalized logistic function","modified Stannard function","kernel density estimation","Bayesian classifier","biomarker trajectories"],"falsifier":"Fit the model twice using disjoint biomarker sets, say CSF and PET versus MRI and cognitive tests, and compare the DPS assigned to each patient-visit; if the two estimates disagree by more than measurement noise, or if adding a quadratic term in age such as $s=\\alpha t+\\beta t^2$ yields a materially lower validation loss, then the single-axis linear DPS assumption fails and the inflection-point ordering is not trustworthy.","tokens_in":19017,"feed_emoji":"🧠","tokens_out":7932,"duration_ms":73361,"temperature":0.7,"pith_summary":"The paper claims that Alzheimer's disease progression can be captured by a single latent disease progression score per patient, with every biomarker following a monotone S-shaped curve of that score, and that this simple parametric picture is accurate enough to predict biomarker values and clinical status from age. The authors introduce a modified Stannard logistic function and fit it jointly across biomarkers using robust M-estimation, which suppresses the influence of outliers. On ADNI test data the model reaches an average normalized mean absolute error of 0.991 across 16 biomarkers and a multiclass AUC of 0.934 in clinical status classification, compared with 1.552 NMAE for the analogous Jedynak model. Trained on ADNI, it also transfers to the NACC cohort with NMAE 1.182 and AUC 0.929, suggesting the learned disease timeline is not specific to one cohort. The inflection points of the fitted curves give a temporal ordering of biomarkers, with CSF, PET, and an early memory test preceding MRI and later cognitive scales.","feed_headline":"0.991: one disease score fits Alzheimer's biomarker curves","feed_subtitle":"Robust M-estimation plus the modified Stannard curve beats earlier Alzheimer's progression models.","key_machinery":"The load-bearing machinery is the modified Stannard logistic function, $$g(s;\\$\\theta$)=\\left[1+\\tfrac{1}{\\gamma}$e^{{-\\frac{b}}${\\gamma}(s-c)}\\right]^{-\\gamma},$$ a flexible S-shaped curve whose asymmetry parameter $\\gamma$ lets the model describe slow-then-fast or fast-then-slow biomarker change while keeping the inflection point at $c$; and the robust fitting scheme, which alternates between estimating biomarker parameters $\\theta_k$ and subject parameters $(\\alpha_i,\\beta_i)$ under the logistic-loss M-estimator $\\rho(r)=\\ln\\cosh(r)$, down-weighting large residuals so outliers do not distort the curves. Bootstrap resampling quantifies the estimates, inflection points provide a temporal ordering of biomarkers, and kernel density estimates of the DPS distributions feed a Bayesian classifier.","core_discovery":"The paper's claim is that a parametric, robust extension of the earlier Jedynak model captures AD progression accurately. Concretely, each subject's age $t_{i,j}$ is mapped to a progression score $s_{i,j}=\\alpha_i t_{i,j}+\\beta_i$, each biomarker $k$ is modeled as a modified Stannard function $f(s;\\theta_k)$ with an asymmetry parameter $\\gamma$, and all parameters are fit jointly by minimizing $\\sum_{i,j,k} w_i \\rho\\!\\left((y_{i,j,k}-f(s_{i,j};\\theta_k))/\\sigma_k\\right)$ with the logistic loss $\\rho(r)=\\ln\\cosh(r)$. On the ADNI test set the model attains an average normalized MAE of 0.991 across 16 biomarkers and a multiclass AUC of 0.934 for CN/MCI/AD classification (0.931 before ensemble bagging), compared with NMAE 1.552 for the analogous Jedynak model. The same model, trained on ADNI, transfers to the NACC cohort with NMAE 1.182 and AUC 0.929, and the inflection points order CSF, PET, and an early memory test before MRI and later cognitive tests.","pith_inferences":["A testable consequence of the single-axis assumption: if progression is truly one-dimensional, the DPS estimated from one biomarker subset should agree with the DPS estimated from another; disagreements would reveal that Alzheimer's has more than one temporal dimension.","The fact that an asymmetric curve (modified Stannard) beats symmetric Verhulst and Richards variants suggests biomarker dynamics are not symmetric around the inflection point; a follow-up could model the asymmetry parameter $\\gamma$ per biomarker as a marker of early versus late acceleration.","Robust M-estimation could be repurposed as an outlier detector: large residuals under logistic loss would flag individual visits as suspect, for instance measurement errors or atypically fast progressors—an application the paper does not develop.","The cross-cohort result implies a strategy for smaller cohorts: train the parametric template on a large, richly measured cohort and then calibrate only per-subject progression rates on a smaller cohort, rather than refitting all curves."],"forward_implications":["With age as the only input, the fitted model predicts all 16 modeled ADNI biomarker values—volumetric MRI, PET, CSF, and cognitive scores—on a common progression axis.","The inflection-point ordering implies a fixed sequence of biomarker abnormality: amyloid beta, amyloid PET, and the RAVLT memory test turn early, MRI atrophy follows, and cognitive scales turn last; the paper uses this ordering for staging.","The Bayesian classifier built on DPS densities separates cognitively normal, MCI, and AD with multiclass AUC around 0.93, so the same progression score used for curve fitting also carries diagnostic information.","Since an ADNI-trained model predicts NACC values with only moderately higher error, the fitted disease timeline generalizes across cohorts and across different sets of available biomarkers.","The model needs only two distinct visits per subject and tolerates missing biomarker measurements, so it can be applied to sparse clinical datasets."],"supporting_citations":[{"why":"Defines the original parametric AD progression model—linear age-to-DPS mapping, sigmoid fitting, alternating least squares—that the proposed method extends and directly compares against.","marker":"Jedynak et al. (2012, 2015)"},{"why":"Provides the base Stannard growth function, which the paper modifies into the novel 'modified Stannard' logistic used for biomarker curves.","marker":"Stannard et al. (1985)"},{"why":"Supplies the M-estimation framework and the logistic-loss type estimators used for robust curve fitting.","marker":"Holland and Welsch (1977)"},{"why":"Gives the hypothetical sigmoidal biomarker cascade of AD that motivates modeling biomarkers as monotone functions of disease progression and supplies the expected ordering.","marker":"Jack Jr et al. (2010)"},{"why":"A state-of-the-art quantitative template model used for head-to-head comparison and whose data splits and benchmark the paper adopts.","marker":"Bilgel and Jedynak (2019)"},{"why":"Defines the TADPOLE challenge and provides best-reported results that the proposed model is compared against.","marker":"Marinescu et al. (2020)"},{"why":"Provides the residual and analysis-of-covariance approach used to correct MRI volumes for head size and cohort or software differences.","marker":"O'Brien et al. (2011)"},{"why":"Establishes kernel density estimation, used to build class-conditional DPS likelihoods for the Bayesian classifier.","marker":"Parzen (1962)"},{"why":"Describes the ADNI cohort used for the main experiments.","marker":"Petersen et al. (2010)"}],"fun_headline_variants":["Modified Stannard curve maps Alzheimer's progression with 0.934 AUC","Robust fit of a new curve tracks Alzheimer's progression across cohorts","One disease score from robust M-estimation orders Alzheimer's biomarkers","Parametric model robustly fits Alzheimer's biomarker dynamics with 0.991 NMAE","New Stannard function robustly orders Alzheimer's biomarkers in disease course"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model assumes a patient's whole disease state is one number—the disease progression score—that grows linearly with age and that every biomarker is a monotone S-shaped function of that same number; if real Alzheimer's trajectories are multidimensional, or if progression speed varies with disease stage rather than age, the fitted curves and the ordering derived from their inflection points are not valid.","fun_headline_variants_meta":{"raw":{"variants":["Modified Stannard curve maps Alzheimer's progression with 0.934 AUC","Robust fit of a new curve tracks Alzheimer's progression across cohorts","One disease score from robust M-estimation orders Alzheimer's biomarkers","Parametric model robustly fits Alzheimer's biomarker dynamics with 0.991 NMAE","New Stannard function robustly orders Alzheimer's biomarkers in disease course"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00089,"raw_usage":{"total_tokens":3920,"prompt_tokens":1107,"completion_tokens":2813,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":723,"completion_tokens_details":{"reasoning_tokens":2715}},"tokens_in":723,"tokens_out":2813,"duration_ms":19668,"temperature":1.0,"reasoning_tokens":2715,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:17:14.032947+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fit the model twice using disjoint biomarker sets, say CSF and PET versus MRI and cognitive tests, and compare the DPS assigned to each patient-visit; if the two estimates disagree by more than measurement noise, or if adding a quadratic term in age such as $s=\\alpha t+\\beta t^2$ yields a materially lower validation loss, then the single-axis linear DPS assumption fails and the inflection-point ordering is not trustworthy.","supporting_citations":[{"cited_title":", author Lang, A","cited_arxiv_id":null,"evidence_quote":"Defines the original parametric AD progression model—linear age-to-DPS mapping, sigmoid fitting, alternating least squares—that the proposed method extends and directly compares against."},{"cited_title":", author Williams, A","cited_arxiv_id":null,"evidence_quote":"Provides the base Stannard growth function, which the paper modifies into the novel 'modified Stannard' logistic used for biomarker curves."},{"cited_title":", author Welsch, R.E","cited_arxiv_id":null,"evidence_quote":"Supplies the M-estimation framework and the logistic-loss type estimators used for robust curve fitting."},{"cited_title":", author Knopman, D.S","cited_arxiv_id":null,"evidence_quote":"Gives the hypothetical sigmoidal biomarker cascade of AD that motivates modeling biomarkers as monotone functions of disease progression and supplies the expected ordering."},{"cited_title":", author Jedynak, B.M","cited_arxiv_id":null,"evidence_quote":"A state-of-the-art quantitative template model used for head-to-head comparison and whose data splits and benchmark the paper adopts."},{"cited_title":", author Ziegler, D.A","cited_arxiv_id":null,"evidence_quote":"Provides the residual and analysis-of-covariance approach used to correct MRI volumes for head size and cohort or software differences."},{"cited_title":", year 1962","cited_arxiv_id":null,"evidence_quote":"Establishes kernel density estimation, used to build class-conditional DPS likelihoods for the Bayesian classifier."},{"cited_title":", author Aisen, P","cited_arxiv_id":null,"evidence_quote":"Describes the ADNI cohort used for the main experiments."}],"review_version":1}