{"id":"00966724-d15a-4d73-a0bb-a21abd8af3e7","arxiv_id":"2509.17661","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A comparator loss trains speech models to produce ordinal severity scores for NDD progression that distinguish patients from controls and correlate with unseen clinical annotations.","lead":"This paper introduces a comparator loss for training speech models to output severity scores that respect ordering from diagnoses, clinical metrics, or recording chronology in neurodegenerative disease monitoring. A smart generalist might read it for its potential to enable continuous, non-invasive tracking of disease progression using voice data with limited labels.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Ordering signals (diagnosis/chronology) may yield only group separation rather than a continuous progression score","rationale":"The reader's weakest assumption directly identifies the same risk. With full text now available the concern remains load-bearing because the abstract-level claim does not yet demonstrate that the continuous score generalizes beyond the supplied orderings; the proposed partial-correlation check would settle whether the generalization holds.","tokens_in":1657,"tokens_out":279,"duration_ms":19871,"concrete_test":"Compute partial Spearman correlation between the learned severity scores and ALSFRS-R (or therapist ratings) after controlling for the binary diagnosis label used in training; if the partial correlation drops below 0.3 or loses significance, the scores largely reflect the training ordering rather than independent severity.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires that comparator loss on diagnosis, clinical-score, or chronological orderings produces scores that meaningfully track disease progression and correlate with held-out metrics such as ALSFRS-R. If the training orderings are dominated by binary diagnosis labels or coarse clinical bins, the loss can be satisfied by a near-binary embedding that separates NDD from controls without enforcing fine-grained, monotonic progression within subjects. The abstract reports correlation with unseen annotations, yet provides no evidence that this correlation survives after regressing out the training ordering cues themselves.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript introduces the comparator loss, an ordinal contrastive loss that trains a model to output a continuous severity score from speech recordings for neurodegenerative disease monitoring. Ordering constraints are derived from diagnosis labels, clinical scores, or recording chronology; the resulting scores are shown to separate NDD patients from controls and to correlate with held-out annotations such as ALSFRS-R and speech-language therapist ratings.","tokens_in":1784,"tokens_out":442,"duration_ms":35664,"significance":"If the central claim holds after the requested clarifications, the work would offer a practical route to leverage lightly annotated speech data for progression tracking, allowing disparate clinical signals to be combined in small health datasets and potentially supporting treatment planning and drug trials.","major_comments":[{"comment":"Abstract: the claim that scores 'correlate with annotations not observed in training, such as ALSFRS-R' is presented without any numerical correlation values, p-values, confidence intervals, or controls that regress out the training ordering cues (diagnosis or chronology); this leaves open whether the correlation reflects genuine progression information or merely group separation.","section":"Abstract"},{"comment":"Methods: the comparator loss is introduced only conceptually ('ensures scores obey an ordering relation') with no explicit equation, loss formulation, or comparison to standard contrastive or ranking losses; without this, it is impossible to verify whether the loss enforces fine-grained monotonicity within subjects or can be satisfied by a near-binary embedding when orderings are coarse.","section":"Methods"}],"minor_comments":[{"comment":"The abstract would benefit from a short statement of dataset size, number of subjects, and recording conditions to allow readers to gauge the scale of the reported distinctions and correlations.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads as an early draft; the absence of equations, ablations, and statistical detail suggests the authors may have more results that were omitted. The topic fits the eess.AS scope but would be strengthened by explicit comparison to existing ordinal or ranking losses in the speech-health literature."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive and detailed feedback. The comments have helped us identify areas where additional clarity and quantitative support will strengthen the manuscript. We respond to each major comment below and have revised the manuscript to address the points raised.","responses":[{"response":"We agree that the abstract would be strengthened by including quantitative results. The results section reports Pearson correlations, p-values, and confidence intervals between the severity scores and held-out annotations such as ALSFRS-R. To address the concern about training cues, we have added a control analysis that regresses out diagnosis labels and chronological order information before computing the correlations; the partial correlations with the unseen clinical annotations remain statistically significant. We will revise the abstract to include these numerical values along with a brief description of the control analysis.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the claim that scores 'correlate with annotations not observed in training, such as ALSFRS-R' is presented without any numerical correlation values, p-values, confidence intervals, or controls that regress out the training ordering cues (diagnosis or chronology); this leaves open whether the correlation reflects genuine progression information or merely group separation."},{"response":"We thank the referee for this observation. We will expand the Methods section to provide the explicit mathematical formulation of the comparator loss (a margin-based pairwise ranking objective applied according to the available ordering constraints) together with a direct comparison to standard contrastive losses and ranking losses. The revised text will also include an explanation and example showing that, when within-subject chronological orderings are supplied, the loss enforces fine-grained monotonicity rather than permitting a near-binary solution.","revision_made":"yes","referee_comment":"[Methods] Methods: the comparator loss is introduced only conceptually ('ensures scores obey an ordering relation') with no explicit equation, loss formulation, or comparison to standard contrastive or ranking losses; without this, it is impossible to verify whether the loss enforces fine-grained monotonicity within subjects or can be satisfied by a near-binary embedding when orderings are coarse."}],"tokens_in":1247,"tokens_out":447,"duration_ms":71957,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing to know is that this paper puts forward a comparator loss to turn speech data into a continuous severity score for neurodegenerative disease monitoring. The loss enforces ordering based on diagnosis, clinical scores, or recording chronology, and the abstract reports that the resulting scores distinguish patients from healthy controls while correlating with annotations like ALSFRS-R and speech therapist ratings that were not used in training.","headline":"Comparator loss uses ordering from diagnosis or chronology to build speech-based severity scores that separate NDD patients from controls and correlate with some held-out clinical measures, but the abstract leaves the loss details and independence from training signals unclear.","tokens_in":2293,"tokens_out":165,"would_cite":false,"duration_ms":33036,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[{"relation":"unclear","rs_module":"IndisputableMonolith/Cost/FunctionalEquation.lean","rs_theorem":"washburn_uniqueness_aczel","paper_passage":"the comparator loss ... J = max((f θ(a)−f θ(b)) +ϵ,0) ... ensures scores follow an ordering relation, which can be based on diagnosis, clinically annotated scores, or simply the chronological order of the recordings"},{"relation":"unclear","rs_module":"IndisputableMonolith/Foundation/ArithmeticFromLogic.lean","rs_theorem":null,"paper_passage":"We evaluated our proposed models based on their ability to affirmatively track the progression of patients with motor neuron disease (MND), the correlation of their output with clinical annotations such as ALSFRS-R"}],"headline":"Comparator loss is a standard hinge-style ordinal contrastive objective unrelated to RS cost or ordering machinery","alignment":"orthogonal","rationale":"The paper introduces a hinge loss J = max((f(a) - f(b)) + ε, 0) that enforces ordering on scalar scores derived from diagnosis, clinical bins or chronology. This is conventional ML ordinal regression for speech biomarkers in MND/ALS monitoring. RS derives a unique reciprocal cost J(x) = ½(x + x⁻¹) - 1, φ-ladders, 8-tick periodicity and parameter-free constants from a single distinction; none of these structures appear in the comparator loss, the training orderings, or the evaluation metrics (ALSFRS-R correlation, progression gradients). The domain (eess.AS health monitoring) lies outside RS scope.","tokens_in":45724,"confidence":"high","tokens_out":377,"duration_ms":13236,"cache_read_input_tokens":38528,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A comparator loss trains speech models to output continuous severity scores that respect clinical orderings and correlate with unseen disease measures.","keywords":["comparator loss","severity score","speech monitoring","neurodegenerative disease","ordinal contrastive loss","ALSFRS-R","health progression"],"falsifier":"A held-out longitudinal speech dataset in which the model scores fail to rise with documented clinical worsening or show zero correlation with later ALSFRS-R values would falsify the central claim.","tokens_in":2563,"feed_emoji":"🗣️","tokens_out":408,"duration_ms":35597,"temperature":0.7,"pith_summary":"The paper proposes training a model on speech recordings to produce a scalar severity score by minimizing a comparator loss that enforces ordering constraints. These constraints can come from diagnosis labels, existing clinical scores, or simply the chronological sequence of recordings. The resulting scores distinguish subjects with neurodegenerative diseases from healthy controls and correlate with independent clinical annotations such as ALSFRS-R and ratings from speech and language therapists, even though those annotations were never shown during training. The method is designed to make fuller use of small, lightly annotated health datasets by combining signals across different metrics through shared orderings.","feed_headline":"Comparator loss yields speech severity scores for disease tracking","feed_subtitle":"Scores trained only on orderings distinguish patients and match unseen clinical ratings such as ALSFRS-R.","key_machinery":"The comparator loss, an ordinal contrastive loss that penalizes pairs whose assigned scores violate a known ordering relation.","core_discovery":"Minimizing the comparator loss on speech data yields a continuous severity score whose values obey ordering relations derived from diagnosis, clinical scores, or recording chronology, and these scores generalize to predict annotations not present in the training data.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Comparator loss derives ordered severity scores from speech","Ordinal contrastive loss produces continuous severity scores","Comparator loss generalizes to predict unseen clinical ratings","Severity scores via comparator loss match ALSFRS-R annotations"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Ordering relations derived from diagnosis or recording times supply enough signal to learn a continuous severity score that reflects actual disease progression beyond those orderings.","fun_headline_variants_meta":{"raw":{"variants":["Comparator loss derives ordered severity scores from speech","Ordinal contrastive loss produces continuous severity scores","Comparator loss generalizes to predict unseen clinical ratings","Severity scores via comparator loss match ALSFRS-R annotations"]},"model":"grok-4.3","cost_usd":0.008988,"raw_usage":{"total_tokens":3901,"prompt_tokens":558,"num_sources_used":0,"completion_tokens":56,"cost_in_usd_ticks":89878000,"prompt_tokens_details":{"text_tokens":558,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3287,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":558,"tokens_out":56,"duration_ms":37542,"temperature":1.0,"reasoning_tokens":3287,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-18T14:59:12.723897+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A held-out longitudinal speech dataset in which the model scores fail to rise with documented clinical worsening or show zero correlation with later ALSFRS-R values would falsify the central claim.","supporting_citations":[],"review_version":1}