Pith. sign in

REVIEW 2 major objections 1 minor 28 references

Comparator Loss: An Ordinal Contrastive Loss to Derive a Severity Score for Speech-based Health Monitoring

T0 review · 2 major / 1 minor · reviewed 2026-05-18 · grok-4.3

Challenge this review Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A comparator loss trains speech models to output continuous severity scores that respect clinical orderings and correlate with unseen disease measures.

desk verdict Comparator loss uses ordering from diagnosis or chronology to build speech-based severity scores that separate NDD patients from controls and correlate with some held-out clinical measures, but the abstract leaves the loss details and independence from training signals unclear. read the letter →

arxiv 2509.17661 v2 submitted 2025-09-22 eess.AS cs.SD

Jacob J Webber , Oliver Watts , Lovisa Wihlborg , Johnny Tam , Christine Weaver , Suvankar Pal , Siddharthan Chandran , Cassia Valentini-Botinhao This is my paper · ORCID
classification eess.AScs.SD
keywords comparatorlossseverityscorespeechmonitoringneurodegenerativediseaseordinalcontrastiveALSFRS-Rhealthprogression
checked against Cost.FunctionalEquationFoundation.ArithmeticFromLogic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes training a model on speech recordings to produce a scalar severity score by minimizing a comparator loss that enforces ordering constraints. These constraints can come from diagnosis labels, existing clinical scores, or simply the chronological sequence of recordings. The resulting scores distinguish subjects with neurodegenerative diseases from healthy controls and correlate with independent clinical annotations such as ALSFRS-R and ratings from speech and language therapists, even though those annotations were never shown during training. The method is designed to make fuller use of small, lightly annotated health datasets by combining signals across different metrics through shared orderings.

What carries the argument

The comparator loss, an ordinal contrastive loss that penalizes pairs whose assigned scores violate a known ordering relation.

What would settle it

A held-out longitudinal speech dataset in which the model scores fail to rise with documented clinical worsening or show zero correlation with later ALSFRS-R values would falsify the central claim.

Watch

Extended reading notes

Core claim

Minimizing the comparator loss on speech data yields a continuous severity score whose values obey ordering relations derived from diagnosis, clinical scores, or recording chronology, and these scores generalize to predict annotations not present in the training data.

Load-bearing premise

Ordering relations derived from diagnosis or recording times supply enough signal to learn a continuous severity score that reflects actual disease progression beyond those orderings.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The manuscript introduces the comparator loss, an ordinal contrastive loss that trains a model to output a continuous severity score from speech recordings for neurodegenerative disease monitoring. Ordering constraints are derived from diagnosis labels, clinical scores, or recording chronology; the resulting scores are shown to separate NDD patients from controls and to correlate with held-out annotations such as ALSFRS-R and speech-language therapist ratings.

Significance. If the central claim holds after the requested clarifications, the work would offer a practical route to leverage lightly annotated speech data for progression tracking, allowing disparate clinical signals to be combined in small health datasets and potentially supporting treatment planning and drug trials.

major comments (2)
  1. [Abstract] Abstract: the claim that scores 'correlate with annotations not observed in training, such as ALSFRS-R' is presented without any numerical correlation values, p-values, confidence intervals, or controls that regress out the training ordering cues (diagnosis or chronology); this leaves open whether the correlation reflects genuine progression information or merely group separation.
  2. [Methods] Methods: the comparator loss is introduced only conceptually ('ensures scores obey an ordering relation') with no explicit equation, loss formulation, or comparison to standard contrastive or ranking losses; without this, it is impossible to verify whether the loss enforces fine-grained monotonicity within subjects or can be satisfied by a near-binary embedding when orderings are coarse.
minor comments (1)
  1. [Abstract] The abstract would benefit from a short statement of dataset size, number of subjects, and recording conditions to allow readers to gauge the scale of the reported distinctions and correlations.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. The comments have helped us identify areas where additional clarity and quantitative support will strengthen the manuscript. We respond to each major comment below and have revised the manuscript to address the points raised.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the claim that scores 'correlate with annotations not observed in training, such as ALSFRS-R' is presented without any numerical correlation values, p-values, confidence intervals, or controls that regress out the training ordering cues (diagnosis or chronology); this leaves open whether the correlation reflects genuine progression information or merely group separation.

    Authors: We agree that the abstract would be strengthened by including quantitative results. The results section reports Pearson correlations, p-values, and confidence intervals between the severity scores and held-out annotations such as ALSFRS-R. To address the concern about training cues, we have added a control analysis that regresses out diagnosis labels and chronological order information before computing the correlations; the partial correlations with the unseen clinical annotations remain statistically significant. We will revise the abstract to include these numerical values along with a brief description of the control analysis. revision: yes

  2. Referee: [Methods] Methods: the comparator loss is introduced only conceptually ('ensures scores obey an ordering relation') with no explicit equation, loss formulation, or comparison to standard contrastive or ranking losses; without this, it is impossible to verify whether the loss enforces fine-grained monotonicity within subjects or can be satisfied by a near-binary embedding when orderings are coarse.

    Authors: We thank the referee for this observation. We will expand the Methods section to provide the explicit mathematical formulation of the comparator loss (a margin-based pairwise ranking objective applied according to the available ordering constraints) together with a direct comparison to standard contrastive losses and ranking losses. The revised text will also include an explanation and example showing that, when within-subject chronological orderings are supplied, the loss enforces fine-grained monotonicity rather than permitting a near-binary solution. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity in the derivation chain

full rationale

The paper introduces the comparator loss as a training objective explicitly constructed to enforce ordering relations drawn from diagnosis labels, clinical scores, or chronological order. The severity score is defined as the scalar output of a model trained under this loss. This construction is intentional and transparent rather than a hidden reduction. The central empirical claims—distinguishing NDD subjects from controls and correlating with held-out annotations such as ALSFRS-R and speech-therapist ratings—are evaluated on data or metrics not used to supply the training orderings. Because these correlations constitute independent, falsifiable evidence outside the fitted orderings, the reported results do not reduce to the inputs by construction. No self-citations, uniqueness theorems, or ansatzes are invoked as load-bearing steps in the provided text. The work is therefore a standard supervised embedding method whose validity rests on external validation rather than definitional equivalence.

Assumptions & free parameters 0 free parameters · 1 assumptions · 2 invented entities

The central claim rests on the domain assumption that available ordering information reflects underlying disease severity and that a model can generalize this to a continuous score validated on unseen metrics.

assumptions (1)
  • domain assumption Ordering relations based on diagnosis, clinical scores, or chronological order reflect true underlying disease progression.
    This premise is invoked to define the comparator loss and train the severity score model.
invented entities (2)
  • comparator loss
    purpose: Training objective that enforces ordinal relations for severity scoring
    Newly proposed loss function in this work.
  • severity score
    purpose: Continuous measure of NDD progression derived from speech
    Output of the model trained with the comparator loss.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Comparator Loss: An Ordinal Contrastive Loss to Derive a Severity Score for Speech-based Health Monitoring." pith.science (2026). https://pith.science/paper/2509.17661

@misc{pith2026250917661,
  author       = {Pith},
  title        = {Pith review of: Comparator Loss: An Ordinal Contrastive Loss to Derive a Severity Score for Speech-based Health Monitoring},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2509.17661}},
  note         = {Machine review of arXiv:2509.17661}
}
read the original abstract

Monitoring the progression of neurodegenerative disease (NDD) has important applications in planning treatment and evaluating new medications. Whereas much work has focused on discriminating patients from healthy controls, or predicting real-world health metrics, we propose a novel measure of disease progression: the severity score, derived from a model trained to minimize what we call the comparator loss. This loss ensures scores obey an ordering relation, based on diagnosis, clinical scores, or simply chronological order of recordings. The proposed comparator loss-based system has the potential to incorporate information from disparate health metrics, critical for making full use of small health-related datasets. We show that a model trained on lightly annotated data is capable of distinguishing between subjects with NDDs and healthy controls. Our score also correlates with annotations not observed in training, such as ALSFRS-R and those of speech and language therapists.

Discussion (0). Continue with ORCID to comment.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

  • IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear
    ?
    unclear

    Relation between the paper passage and the cited Recognition theorem.

    the comparator loss ... J = max((f θ(a)−f θ(b)) +ϵ,0) ... ensures scores follow an ordering relation, which can be based on diagnosis, clinically annotated scores, or simply the chronological order of the recordings

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

28 extracted references · 28 canonical work pages

  1. [1]

    INTRODUCTION Neurodegenerative disorders are becoming more prevalent globally, posing a significant health challenge. They include progressive and fatal diagnoses affecting cognition such as Alzheimer’s disease, and neuromuscular disorders such as motor neuron disease (MND), the most common subtype of which is amyotrophic lateral sclero- sis (ALS). There ...

    work page Pith review arXiv 2025

  2. [2]

    Category learning Categorical losses, such as cross-entropy, are well-established in the field of machine learning and commonly used in medical tasks

    BACKGROUND 2.1. Category learning Categorical losses, such as cross-entropy, are well-established in the field of machine learning and commonly used in medical tasks. These make no assumption on the ordering of classes, meaning there is no inductive bias encoded in the model that categories are sequential. However, cross-entropy has also been applied to o...

    work page

  3. [3]

    Their proposed model is trained to learn a low-dimensional manifold from images

    where authors introduce a contrastive loss for dimensionality re- duction. Their proposed model is trained to learn a low-dimensional manifold from images. Pairs of images that are of similar categories are mapped to nearby points in that manifold. To prevent the repre- sentation space collapsing, a contrastive term is introduced, forcing image pairs from...

    work page

  4. [4]

    Model Fig

    METHOD 3.1. Model Fig. 1 illustrates the general structure of the proposed comparator model. The model is composed of an encoder that outputs an em- bedding given a speech waveform and a scoring network that takes this embedding and converts it into a scalar valuef θ(.). We refer to this value as the ‘score’ as it can be used as a proxy for a sever- ity s...

    work page

  5. [5]

    Data We used the Anne Rowling Neurological Speech Corpus [20] for training and evaluation of the proposed system

    EV ALUA TION 4.1. Data We used the Anne Rowling Neurological Speech Corpus [20] for training and evaluation of the proposed system. The corpus con- tains speech recordings of 780 individuals, grouped according to their documented diagnosis. We used data from the healthy control group (HC) of 170 individuals and the motor neuron disease (MND) group of 103....

    work page

  6. [6]

    While it is not yet possible to use the scores analysed here to confirm progression of disease using longitudinal data, the system compares favourably with strong baselines

    CONCLUSIONS A new loss has been proposed; the predictions of models trained with it correlate significantly with clinical judgments. While it is not yet possible to use the scores analysed here to confirm progression of disease using longitudinal data, the system compares favourably with strong baselines. As the design of the loss enables the exploita- ti...

    work page

  7. [7]

    Alize J Ferrari et al., “Global incidence, prevalence, years lived with disability, disability-adjusted life-years, and healthy life expectancy for 371 diseases and injuries in 204 countries and territories and 811 subnational locations, 1990–2021: a sys- tematic analysis for the Global Burden of Disease Study 2021,” The Lancet, vol. 403, no. 10440, pp. 2...

    work page 1990

  8. [8]

    A systematic review and narrative analy- sis of digital speech biomarkers in motor neuron disease,

    Molly Bowden et al., “A systematic review and narrative analy- sis of digital speech biomarkers in motor neuron disease,”NPJ digital medicine, vol. 6, no. 1, pp. 228, 2023

    work page 2023

Show all 28 references
  1. [9]

    Artificial intelligence, speech, and language processing approaches to monitoring Alzheimer’s disease: a systematic review,

    Sofia De la Fuente Garcia, Craig W Ritchie, and Saturnino Luz, “Artificial intelligence, speech, and language processing approaches to monitoring Alzheimer’s disease: a systematic review,”Journal of Alzheimer’s Disease, vol. 78, no. 4, pp. 1547–1574, 2020

    2020

  2. [10]

    Alzheimer’s dementia recognition through spontaneous speech: The ADReSS Challenge,

    Saturnino Luz et al., “Alzheimer’s dementia recognition through spontaneous speech: The ADReSS Challenge,” in Proc. Interspeech, Shanghai, China, 2020

    2020

  3. [11]

    The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respi- ratory function,

    Jesse M Cedarbaum et al., “The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respi- ratory function,”Journal of the neurological sciences, vol. 169, no. 1-2, pp. 13–21, 1999

    1999

  4. [12]

    ALS-specific cognitive and behavior changes associated with advancing disease stage in ALS,

    Christopher Crockford et al., “ALS-specific cognitive and behavior changes associated with advancing disease stage in ALS,”Neurology, vol. 91, no. 15, pp. e1370–e1380, 2018

    2018

  5. [13]

    A brief cognitive test battery to differentiate Alzheimer’s disease and frontotemporal demen- tia,

    Pavagada S Mathuranath et al., “A brief cognitive test battery to differentiate Alzheimer’s disease and frontotemporal demen- tia,”Neurology, vol. 55, no. 11, pp. 1613–1620, 2000

    2000

  6. [14]

    ALSFRS and appel ALS scores: discordance with disease progression,

    Andrei V oustianiouk et al., “ALSFRS and appel ALS scores: discordance with disease progression,”Muscle & Nerve: Offi- cial Journal of the American Association of Electrodiagnostic Medicine, vol. 37, no. 5, pp. 668–672, 2008

    2008

  7. [15]

    Using the ALSFRS-R in multicentre clinical trials for amyotrophic lateral sclerosis: potential limi- tations in current standard operating procedures,

    Jaap NE Bakers et al., “Using the ALSFRS-R in multicentre clinical trials for amyotrophic lateral sclerosis: potential limi- tations in current standard operating procedures,”Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration, vol. 23, no. 7-8, pp. 500–507, 2022

    2022

  8. [16]

    A further rasch study confirms that ALSFRS-R does not conform to fundamental measurement requirements,

    Franco Franchignoni, Jessica Mandrioli, Andrea Giordano, Salvatore Ferro, and ERRALS Group, “A further rasch study confirms that ALSFRS-R does not conform to fundamental measurement requirements,”Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration, vol. 16, no. 5-6, ...

    2015

  9. [17]

    Longitudinal modeling of de- pression shifts using speech and language,

    Paula Andrea P ´erez-Toro et al., “Longitudinal modeling of de- pression shifts using speech and language,” inProc. ICASSP, 2024, pp. 12021–12025

    2024

  10. [18]

    The PHQ-8 as a measure of current depression in the general population,

    Kurt Kroenke, Tara W. Strine, Robert L. Spitzer, Janet B.W. Williams, Joyce T. Berry, and Ali H. Mokdad, “The PHQ-8 as a measure of current depression in the general population,” Journal of Affective Disorders, vol. 114, no. 1, pp. 163–173, 2009

    2009

  11. [19]

    Efficient neural audio synthesis,

    Nal Kalchbrenner, Erich Elsen, Karen Simonyan, Seb Noury, Norman Casagrande, Edward Lockhart, Florian Stim- berg, Aaron van den Oord, Sander Dieleman, and Koray Kavukcuoglu, “Efficient neural audio synthesis,” inProc. ICML, 2018, vol. 80, pp. 2410–2419

    2018

  12. [20]

    A neu- ral network approach to ordinal regression,

    Jianlin Cheng, Zheng Wang, and Gianluca Pollastri, “A neu- ral network approach to ordinal regression,” inProc. IJCNN, 2008, pp. 1279–1284

    2008

  13. [21]

    Ensemble of deep neural network models for MOS prediction,

    Marie Kune ˇsov´a, Jind ˇrich Matou ˇsek, Jan Lehe ˇcka, Jan ˇSvec, Josef Mich ´alek, Daniel Tihelka, Martin Bul ´ın, Zdenˇek Hanzl´ıˇcek, and Mark´eta ˇRez´aˇckov´a, “Ensemble of deep neural network models for MOS prediction,” inProc. ICASSP, 2023, pp. 1–5

    2023

  14. [22]

    Contrastive representation learning,

    Lilian Weng, “Contrastive representation learning,”lilian- weng.github.io, May 2021

    2021

  15. [23]

    Dimensionality reduc- tion by learning an invariant mapping,

    R. Hadsell, S. Chopra, and Y . LeCun, “Dimensionality reduc- tion by learning an invariant mapping,” inProc. CVPR, 2006, vol. 2, pp. 1735–1742

    2006

  16. [24]

    FaceNet: A unified embedding for face recognition and clus- tering,

    Florian Schroff, Dmitry Kalenichenko, and James Philbin, “FaceNet: A unified embedding for face recognition and clus- tering,” inProc. CVPR, June 2015

    2015

  17. [25]

    Deep metric learning via lifted struc- tured feature embedding,

    Hyun Oh Song et al., “Deep metric learning via lifted struc- tured feature embedding,” inProc. CVPR, 2016

    2016

  18. [26]

    Anne Rowling Neurological Speech Cor- pus: clinically annotated longitudinal dataset for developing speech biomarkers in neurodegenerative disorders,

    Johnny Tam et al., “Anne Rowling Neurological Speech Cor- pus: clinically annotated longitudinal dataset for developing speech biomarkers in neurodegenerative disorders,” inProc. Interspeech, 2025

    2025

  19. [27]

    Ti- taNet: Neural model for speaker representation with 1d depth- wise separable convolutions and global context,

    Nithin Rao Koluguri, Taejin Park, and Boris Ginsburg, “Ti- taNet: Neural model for speaker representation with 1d depth- wise separable convolutions and global context,” inProc. ICASSP, 2022, pp. 8102–8106

    2022

  20. [28]

    ArcFace: Additive angular margin loss for deep face recognition,

    Jiankang Deng et al., “ArcFace: Additive angular margin loss for deep face recognition,” inProc. CVPR, 2019, pp. 4685– 4694

    2019

Pith tools

Reviewed May 18, 2026 · model on record in the stance chip above.