Probabilistic Textual Time Series Depression Detection

Fabian Schmidt; Seyedehmoniba Ravan; Vladimir Vlassov

REVIEW 3 major objections 7 minor 5 references

Probabilistic Textual Time Series Depression Detection

T0 review · 3 major / 7 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Probabilistic text model predicts depression severity with calibrated uncertainty from clinical interviews.

desk verdict Reasonable probabilistic model for PHQ-8 prediction, but the SOTA claim rests on an input change (WhisperX retranscription) that isn't controlled for. read the letter →

arxiv 2511.04476 v2 pith:5WTHNQYP submitted 2025-11-06 cs.CL

Fabian Schmidt , Seyedehmoniba Ravan , Vladimir Vlassov This is my paper

classification cs.CL

keywords depressiondetectionPHQ-8uncertaintyquantificationtextualtimeseriesprobabilisticneuralnetworkclinicalNLPcalibrationLSTMself-attention

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that depression severity from clinical interviews is best treated as a probabilistic textual time series: each utterance updates a predictive distribution over the PHQ-8 score rather than a single point estimate. By training LSTMs with self-attention and Gaussian or Student-t output heads on negative log-likelihood, the authors obtain a system that is both accurate and uncertainty-aware. On E-DAIC it reports MAE 3.85 and RMSE 4.52, beating previous text-only systems, while producing well-calibrated intervals. The authors see this as evidence that probabilistic modeling and temporal structure belong in clinical NLP, not just point-prediction pipelines.

What carries the argument

The engine is a bidirectional LSTM over utterance embeddings, followed by multi-head self-attention with residual connections, and a probabilistic output head. The Gaussian head predicts mean and variance per timestep; the Student-t head additionally predicts degrees of freedom. Training minimizes negative log-likelihood, optionally reweighted, and seq2seq mode lets confidence be read off at every utterance despite a single session-level target. Attention masks and log-transformed targets support stable training on variable-length interviews.

What would settle it

Run PTTSD and the strongest baselines on the same transcript text — either official E-DAIC transcripts or the re-transcribed version — and check whether the MAE/RMSE gap persists; if PTTSD loses its margin on matched input, the state-of-the-art claim is an artifact of transcription.

Watch

Extended reading notes

Core claim

The central claim is that modeling PHQ-8 regression as a fully probabilistic sequence problem — predicting a Gaussian or Student-t distribution over the score at each utterance (seq2seq) or for the whole interview (seq2one) — yields both competitive point accuracy and calibrated confidence estimates. The authors report state-of-the-art text-only results on E-DAIC (MAE 3.85, RMSE 4.52) and competitive results on DAIC, with an ECE of 0.022 and 66.2% empirical coverage at the nominal 68% level for the Gaussian NLL model. They interpret this as showing that uncertainty estimates track prediction quality, with a Pearson correlation of 0.88 between predicted uncertainty and absolute error.

Load-bearing premise

The system is evaluated on its own re-transcribed text rather than the official transcripts used by all comparison models, so the reported accuracy advantage could come from better transcription, not from the probabilistic architecture.

Editorial extensions

If this is right

If the reported numbers hold, a simple, prompt-free text-only pipeline can match or beat far more elaborate multimodal and prompt-based systems on E-DAIC.
Clinicians could use the predicted interval width as a flag for when the model is unsure, rather than receiving only a point score.
The seq2seq variant shows how confidence evolves during the interview, potentially indicating which parts of a session are most informative.
The ablation implies that self-attention and residual connections are load-bearing for accuracy, while the variance head costs a little raw error in exchange for calibrated uncertainty.
Default Gaussian NLL with balanced weights appears to be the right training objective for sharp and calibrated intervals.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The comparison is not apples-to-apples: all baselines were scored on official transcripts, while PTTSD was scored on automatically re-transcribed audio; if transcription quality is the driver, the state-of-the-art claim may not survive matched transcripts.
The same probabilistic time-series treatment could be applied to other single-session clinical ratings, and the temporal confidence signal could feed adaptive data collection or active learning in digital therapy.
Because the corpus has fewer than 300 sessions and uses a virtual interviewer, calibration quality on genuine clinician-patient speech remains an open, testable question.
The strong uncertainty-error correlation suggests predicted variance could serve as a screening gate, sending only high-uncertainty cases for human review; the paper does not test this.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: the model is sensible and the calibration work is worth a look, but the headline "state-of-the-art" number is not clean. The stress-test note has it right: the paper retranscribes all audio with WhisperX, while the baselines in Table 1 mostly ran on official transcripts or standard Whisper. That changes the input, so the MAE/RMSE gains could come from better transcripts rather than the model. The paper is upfront about the retranscription, which I credit, but it never runs the obvious controls: PTTSD on official transcripts, or at least one baseline on WhisperX text. Without those, the SOTA claim is unsupported.

What's genuinely new: a sequence-to-sequence LSTM with attention that outputs a predictive distribution at every utterance, trained with NLL, for text-only PHQ-8 regression. The three-part calibration analysis (ECE, coverage plots, error-uncertainty correlation) is more thorough than most papers in this area, and the ECE of 0.022 on the DAIC test set is decent if it holds up. The limitations section is candid about the small dataset, the Wizard-of-Oz interviews, and the lack of clinical evaluation. The GitHub link is a good sign, though I haven't verified the code runs.

The soft spots, in proportion: First, the SOTA comparison is the load-bearing issue. Even though Sadeghi et al. used Whisper rather than official transcripts, WhisperX is not Whisper; differences in speaker diarization and alignment can affect what the model sees. This is fixable, but as it stands the comparison conflates transcript quality with model quality. Second, the seq2seq evaluation repeats the same PHQ-8 label at every utterance and then averages per-utterance metrics. That inflates the effective sample size, and it makes the reported r=0.88 between uncertainty and error look stronger than it is—those aren't independent observations. Third, the ablation shows that removing the variance head actually improves MAE and RMSE. The paper frames this as a trade-off, which is fair, but it means the "value of probabilistic modeling" is about calibration, not accuracy. The abstract should say that.

The core idea—uncertainty counts in clinical decision support—still holds. The paper is worth engaging with, but it needs proper controls and a more careful evaluation protocol before its claims can be trusted. I'd send it to reviewers, with the explicit ask that they require the control experiments.

Referee Report

3 major / 7 minor

Summary. The paper introduces PTTSD, a probabilistic framework for predicting PHQ-8 depression severity from utterance-level transcripts, with sequence-to-sequence and sequence-to-one variants built on bidirectional LSTMs, self-attention, residual connections, and Gaussian or Student-t output heads trained with NLL. The authors report state-of-the-art text-only results on E-DAIC (MAE 3.85, RMSE 4.52) and DAIC (MAE 3.55, RMSE 4.77), claim well-calibrated uncertainty via ECE and coverage analysis, and present ablations, hyperparameter sensitivity, and case studies. The manuscript is clearly written and includes a limitations section, but the central SOTA claim rests on an uncontrolled input-text change, and the sequence-to-sequence evaluation protocol is not precisely defined.

Significance. If the SOTA claim were validated, PTTSD would be a useful contribution to clinical NLP: it offers a reproducible, prompt-free, fully probabilistic model with uncertainty estimates, three-run averaged results, public code, and a multi-part calibration analysis. The uncertainty-calibration component is a valuable direction, and the paper's framing around clinical decision support is appropriate. However, the re-transcription of all audio with WhisperX changes the model's input relative to most baselines, so the headline accuracy gains cannot currently be attributed to the model. The sequence-to-sequence evaluation also needs clarification before the numbers in Table 1 can be compared with single-session baselines. The underlying ideas are defensible, but the evidence as presented does not yet establish the paper's strongest claims.

major comments (3)

[§3.2, §4.2, Table 1] The state-of-the-art claim is undermined by an uncontrolled input change. Section 3.2 states that original E-DAIC audio was re-transcribed with WhisperX, and Section 4.1 states that all audio is re-transcribed. PTTSD is therefore evaluated on text that differs from the official E-DAIC/DAIC-WOZ transcripts used by most baselines in Table 1 (with the exception of Sadeghi et al. 2024's Whisper-based variant). No control experiment is reported in which PTTSD is run on the official transcripts, nor are any baselines run on the WhisperX transcripts. The observed gains in MAE/RMSE may thus be due to the re-transcribed input rather than to PTTSD's architecture or probabilistic training. This is load-bearing for the paper's headline contribution; please provide a controlled comparison or explicitly downgrade the claim.
[§3.4–3.5, §4.1, Table 1] The evaluation protocol for the sequence-to-sequence variant is not defined tightly enough for a fair comparison. In seq2seq mode, the session-level PHQ-8 label is used as the target at every utterance, and Section 4.1 says metrics are computed over all valid (non-padded) utterances. This means each participant contributes multiple predictions, and a per-utterance aggregate weights longer interviews more heavily than a per-session baseline. The paper does not state whether Table 1 reports per-utterance MAE/RMSE, a per-participant average of the T predictions, or another aggregation. The same repeated-label structure affects the calibration and error-uncertainty correlation analyses, where per-utterance predictions are treated as independent samples, inflating the effective sample size and significance. Please clarify the aggregation and justify comparability with single-session baselines
[§4.3, Table 3] Ablation results do not support the claim that probabilistic modeling improves accuracy. Removing the variance head and training with MSE yields MAE 5.98 and RMSE 7.21, which are better than the full model's 6.32 and 8.10. The abstract and conclusion state that ablations confirm the value of probabilistic modeling, but Table 3 shows the opposite for point-error metrics. Moreover, the ablation full model is trained for only 20 epochs and reaches MAE 6.32/RMSE 8.10, far worse than the main full model's 3.85/4.52 in Table 1, so the ablation is not performed at a comparable training budget. Please retrain all ablation variants to convergence and separate the effect of loss change from removal of the variance head.

minor comments (7)

[§3.3] The sentence 'We represent each utterance using pretrained sentence encoders.' is duplicated verbatim.
[§4.2, Table 1] For DAIC, PTTSD's RMSE 4.77 is slightly worse than Fang et al.'s 4.76; the statement that PTTSD 'matches the best test RMSE' is imprecise.
[§4.3, Table 2] The text says MSE 'yields the highest test RMSE among all probabilistic losses,' but MSE is not a probabilistic loss and its test RMSE (4.876) is lower than Gaussian NLL (5.0219) and Student-t (5.1488). Please rephrase or correct.
[§4.5, Figure 3] Calibration analysis is reported on the DAIC test set, while the headline SOTA result is on E-DAIC. Please report E-DAIC calibration or justify the choice.
[§E.1, §E.2, Figures 6–7] The uncertainty ranges differ across figures (up to 35 in Figure 7, up to 60 in Figure 3), and Figure 6 contains the placeholder text 'Multiple model sources not available.' Please clarify which configuration/split each figure uses and remove placeholder text.
[References and Table 1] Fang et al. 2023a and 2023b appear to be the same publication; the table and related work use both labels. The related-work table also has a typo, 'Oureshi et al. (2021)' for 'Qureshi et al. (2021)'.
[§3.2, Appendix B.1] Appendix B.1 states that E-DAIC is already a re-transcribed and quality-controlled extension of DAIC-WOZ; Section 3.2 then says the authors re-transcribed the audio again with WhisperX. Please clarify the relationship between the official E-DAIC transcripts, the WhisperX transcripts, and the input actually used for each dataset.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: PTTSD is a standard supervised probabilistic regression pipeline evaluated on held-out data; the WhisperX re-transcription issue is a comparison-validity concern, not a reasoning loop.

full rationale

The paper's derivation chain is a standard supervised-learning pipeline. PTTSD encodes utterances with pretrained sentence encoders, models p(y_t | e_{<=t}) or p(y | e_{1:T}) using an LSTM, self-attention, and residual connections, and trains by negative log-likelihood against ground-truth PHQ-8 scores. There is no equation in which a claimed prediction is defined by the target, and no fitted parameter is renamed as a prediction. Hyperparameters (learning rate, early stopping, loss weights) are selected on the development split, and all headline metrics are reported on held-out test participants (Sections 3.7 and 4.1). The calibration analyses (ECE, coverage, error-uncertainty correlation) are computed from test-set predictions against observed errors and ground-truth values (Sections 4.5 and E.2), not against training targets. No load-bearing self-citation appears: the architecture is 'inspired by (Mandal et al., 2025)', and the encoders (Reimers and Gurevych, 2019; Ji et al., 2022) are external works; no 'uniqueness theorem' is invoked. The main methodological caveat is the WhisperX re-transcription of all audio (Section 3.2), which makes the state-of-the-art comparison against baselines using official transcripts confounded; that is a validity/comparability threat, not circularity. Similarly, the seq2seq variant's use of the same session-level PHQ-8 label at every timestep is a modeling assumption, not an equivalence of prediction and input. Because no circular step can be exhibited from the paper's own equations or self-citation chain, the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. It relies on existing pretrained encoders, a standard LSTM-attention architecture, and a probabilistic output head. The main assumptions are about the adequacy of the text input and the fixedness of the target.

assumptions (3)

domain assumption Pretrained sentence embeddings (all-MiniLM-L6-v2 and MentalBERT) provide useful semantic representations of clinical utterances.
The model is built on top of these embeddings without fine-tuning on clinical data; if the embeddings miss clinical cues, the model cannot recover them.
domain assumption WhisperX transcriptions are faithful to the spoken content and of sufficient quality for PHQ-8 prediction.
All inputs to PTTSD are WhisperX transcripts; errors in transcription could change the estimation. The paper does not validate transcription quality against gold transcripts.
domain assumption PHQ-8 score is stable across the interview and can be predicted from every utterance in the sequence (for seq2seq variant).
The seq2seq model attaches the same session-level PHQ-8 target to every timestep, assuming the severity is constant; this may not reflect temporal dynamics and is an ad-hoc modeling choice.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Probabilistic Textual Time Series Depression Detection." pith.science (2026). https://pith.science/paper/5WTHNQYP

@misc{pith2026251104476,
  author       = {Pith},
  title        = {Pith review of: Probabilistic Textual Time Series Depression Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5WTHNQYP}},
  note         = {Machine review of arXiv:2511.04476}
}

read the original abstract

Accurate and interpretable predictions of depression severity are essential for clinical decision support, yet existing models often lack uncertainty estimates and temporal interpretability. We propose PTTSD, a Probabilistic framework for Depression Detection from clinical interview utterance sequences that predicts PHQ-8 scores while modeling calibrated uncertainty. PTTSD includes sequence-to-sequence and sequence-to-one variants, both combining LSTMs, self-attention, and residual connections with Gaussian or Student's-t output heads trained via negative log-likelihood. The sequence-to-sequence variant enables temporal analysis of how predictive confidence evolves over an interview, despite the target being a single session-level score. Evaluated on E-DAIC and DAIC-WOZ, PTTSD achieves competitive performance among text-only systems (e.g., MAE = 3.85 on E-DAIC, 3.55 on DAIC) and produces well-calibrated prediction intervals. Ablations confirm the value of attention and probabilistic modeling, while a three-part calibration analysis and qualitative case studies highlight the clinical relevance of uncertainty-aware prediction.

Figures

Figures reproduced from arXiv: 2511.04476 by the authors.

**Figure 2.** Ablation results full 50 used in main experiments) to accelerate comparison. Evaluation is performed on the test set using mean absolute error (MAE) and root mean squared error (RMSE). Full experimental details are included in Appendix C. Variant MAE ∆ MAE (%) RMSE ∆ RMSE (%) Full Model 6.32 – 8.10 – - w/o Attention 7.74 +22.48 9.74 +20.24 - w/o Residual 7.19 +13.78 8.96 +10.53 - w/o Variance Head 5.98 −5.37 7.21 −1… view at source ↗

**Figure 3.** Calibration analysis of PTTSD seq2seq on DAIC test set (Gaussian NLL vs. MSE) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 5.** Figure 5: Uncertainty distributions by severity group [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

**Figure 4.** Figure 4: Case studies illustrating prediction, ground [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

**Figure 6.** Figure 6: Sharpness calibration tradeoff ing higher confidence. However, sharpness must be balanced with calibration: a model that is too sharp may be overconfident, while a model that is too broad may be underconfident [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

5 extracted references · 2 linked inside Pith

[2014]

InProceedings of the Ninth International Conference on Language Resources and Evaluation (LREC‘14), pages 3123– 3128, Reykjavik, Iceland

The distress analysis interview corpus of human and computer interviews. InProceedings of the Ninth International Conference on Language Resources and Evaluation (LREC‘14), pages 3123– 3128, Reykjavik, Iceland. European Language Re- sources Association (ELRA). Yanrong Guo, Chenyang Zhu, Shijie Hao, and Richang Hong. 2022. A topic-attentive transformer-bas...

arXiv 2022
[2018]

Wizard-of- Oz

Depression severity estimation from multiple modalities. In2018 IEEE 20th International Con- ference on e-Health Networking, Applications and Services (Healthcom), pages 1–6. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need.Advances in neural inform...

2017
[2019]

InINTERSPEECH 2019, pages 1443– 1447

Detecting depression with word-level multi- modal fusion. InINTERSPEECH 2019, pages 1443– 1447. Misha Sadeghi, Bernhard Egger, Reza Agahi, Robert Richer, Klara Capito, Lydia Helene Rupp, Lena Schindler-Gmelch, Matthias Berking, and Bjoern M. Eskofier. 2023. Exploring the capabilities of a lan- guage model-only approach for depression detection 10 in text ...

2019
[2023]

Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers

Prediction of depression severity based on the prosodic and semantic features with bidirectional lstm and time distributed cnn.IEEE Transactions on Affective Computing, 14(3):2251–2265. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. MTEB: Massive text embedding benchmark. InProceedings of the 17th Conference of the European Chapter...

arXiv 2023
[2024]

Evgeny A

Harnessing multimodal approaches for depres- sion detection using large language models and facial expressions.npj Mental Health Research, 3(1):66. Evgeny A. Stepanov, Stéphane Lathuilière, Sham- mur Absar Chowdhury, Arindam Ghosh, Radu- Lauren¸ tiu Vieriu, Nicu Sebe, and Giuseppe Riccardi

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [2014]

InProceedings of the Ninth International Conference on Language Resources and Evaluation (LREC‘14), pages 3123– 3128, Reykjavik, Iceland

The distress analysis interview corpus of human and computer interviews. InProceedings of the Ninth International Conference on Language Resources and Evaluation (LREC‘14), pages 3123– 3128, Reykjavik, Iceland. European Language Re- sources Association (ELRA). Yanrong Guo, Chenyang Zhu, Shijie Hao, and Richang Hong. 2022. A topic-attentive transformer-bas...

arXiv 2022

[2] [2018]

Wizard-of- Oz

Depression severity estimation from multiple modalities. In2018 IEEE 20th International Con- ference on e-Health Networking, Applications and Services (Healthcom), pages 1–6. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need.Advances in neural inform...

2017

[3] [2019]

InINTERSPEECH 2019, pages 1443– 1447

Detecting depression with word-level multi- modal fusion. InINTERSPEECH 2019, pages 1443– 1447. Misha Sadeghi, Bernhard Egger, Reza Agahi, Robert Richer, Klara Capito, Lydia Helene Rupp, Lena Schindler-Gmelch, Matthias Berking, and Bjoern M. Eskofier. 2023. Exploring the capabilities of a lan- guage model-only approach for depression detection 10 in text ...

2019

[4] [2023]

Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers

Prediction of depression severity based on the prosodic and semantic features with bidirectional lstm and time distributed cnn.IEEE Transactions on Affective Computing, 14(3):2251–2265. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. MTEB: Massive text embedding benchmark. InProceedings of the 17th Conference of the European Chapter...

arXiv 2023

[5] [2024]

Evgeny A

Harnessing multimodal approaches for depres- sion detection using large language models and facial expressions.npj Mental Health Research, 3(1):66. Evgeny A. Stepanov, Stéphane Lathuilière, Sham- mur Absar Chowdhury, Arindam Ghosh, Radu- Lauren¸ tiu Vieriu, Nicu Sebe, and Giuseppe Riccardi