{"id":"a26fe5d2-a388-4ab8-bcea-b6466235e0d6","arxiv_id":"2505.22863","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A multimodal LLM pipeline with Wav2Vec audio and WHO-based Q&A knowledge injection reports small improvements on DAIC-WOZ, but the fusion adds nothing over audio-only and the baseline is cherry-picked.","lead":"This paper applies large language models to predict depression severity from spoken interview audio and transcripts, using the DAIC-WOZ dataset. It claims that injecting psychological knowledge as question-answer pairs improves accuracy, but the reported gains are tiny and the full multimodal model shows no benefit over audio alone.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central empirical claim is contradicted by Table 2 as printed: the full multimodal + knowledge-injection model (MAE 5.356, RMSE 6.713) is identical to both the audio + knowledge-injection and the audio + text variants, so neither fusion nor knowledge injection shows any effect.","rationale":"The reader's REJECT verdict is sound, but my emphasis differs from the reader's flagged weakest assumption. The reader identifies label repetition as the weakest assumption; that is a valid and serious evaluation-protocol concern. However, the paper fails an even more elementary test: its own Table 2 does not show any benefit from the two advertised components in the full model. The only condition under which the central claim could survive is if the printed entries are typos and the code produces different numbers. That is empirically checkable because the code is provided. If corrected numbers do show a fusion/injection benefit, then the label-repetition issue and the cherry-picked baseline become the decisive concerns. My recommendation is therefore unchanged: REJECT as written, with the path to resubmission being a corrected, reproducible result table and a sounder evaluation protocol. I partially agree with the reader because we converge on the same verdict, but I do not treat label repetition as the most load-bearing concern; the self-contradictory result table is.","tokens_in":11255,"tokens_out":5605,"duration_ms":57111,"concrete_test":"Run the released GitHub code on the DAIC-WOZ test partition with the stated seed 42, and recompute all Table 2 rows from saved checkpoints, recording full precision. Then perform a paired bootstrap over participant-level averaged predictions for the three variants 'audio', 'audio+text', and 'audio+text+KI'. If the MAE/RMSE differences among these variants are within rounding error (or the bootstrap intervals include zero), the multimodal and knowledge-injection claims are unsupported; if the rows differ, the printed table is erroneous and corrected numbers plus uncertainty intervals must be provided.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing empirical claim of the paper—that the multimodal LLM pipeline with psychological knowledge injection yields a notable improvement—is contradicted by Table 2 as printed. The full model 'Ours(audio+text+Knowledge Injection)' reports MAE 5.356, RMSE 6.713, which is exactly the same rounded result as 'Ours(audio+Knowledge Injection)' and 'Ours(audio+text)' (both 5.356/6.713). Thus, on the reported metrics, adding text to the audio stream and adding knowledge injection to the full model each change nothing; the full system is also slightly worse than 'Ours(text+Knowledge Injection)' (5.354/6.429) on both metrics. The headline comparison to the AVEC2016 audio baseline (5.72/7.78) omits that the same table lists LSTM (5.70/6.59) and Random Forest (5.71/6.79) baselines; the full model's RMSE of 6.713 is worse than the LSTM's. This is an internal inconsistency, not a question of consensus: if the numbers are reproduced, the central claim fails on the paper's own evidence; if they are typographical, the table must be corrected and re-evaluated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a multimodal pipeline for depression severity regression on the DAIC-WOZ dataset, combining Wav2Vec 2.0 audio embeddings, ASR transcripts, and a psychological knowledge-injection step in which DeepSeek generates question–answer pairs from WHO texts and LLaMA is fine-tuned on them with LoRA. The authors report MAE and RMSE for several ablations, compare them with AVEC 2016 audio and LSTM/Random Forest baselines, and claim notable improvements in both metrics. The paper also includes a self-assessment of the knowledge-injection step using PsycholexLLAMA as both question generator and answer grader.","tokens_in":11560,"tokens_out":3902,"duration_ms":40989,"significance":"If the reported results were valid, the paper would offer a useful practical contribution: a reproducible, code-released LLM-based approach to multimodal depression recognition with a mechanism for injecting domain knowledge. The authors also deserve credit for fixing the random seed, reporting hyperparameters, and performing a structured ablation. However, the central empirical claim is not supported by the manuscript as written. Table 2 shows that the full multimodal model with knowledge injection achieves exactly the same rounded MAE (5.356) and RMSE (6.713) as the audio-only and audio-plus-knowledge variants, so the text stream and the knowledge injection have no measurable effect when audio is present. The evaluation protocol also assigns the same per-participant PHQ-8 score to every clip from that participant, which leaves open the possibility that the model is exploiting participant identity rather than depression-relevant content. These are load-bearing problems: the abstract's 'notable improvement' claim, the conclusion's numerical summary, and the claimed benefit of psychological knowledge injection all rest on numbers that the paper itself does not consistently support.","major_comments":[{"comment":"The central empirical claim is contradicted by Table 2 as printed. 'Ours(audio+text+Knowledge Injection)' reports MAE 5.356 and RMSE 6.713, identical at the reported precision to both 'Ours(audio+text)' and 'Ours(audio+Knowledge Injection)'; consequently, the results show no effect of the text stream and no effect of knowledge injection when audio is present. The full model's RMSE of 6.713 is also worse than the LSTM baseline of 6.59 listed in the same table. In addition, the conclusion's stated improvement of '1.36 on RMSE' corresponds to comparing the AVEC2016 baseline (7.78) with 'Ours(text+Knowledge Injection)' (6.429), not with the full multimodal model, whose RMSE improvement over the AVEC2016 baseline is only 7.78 − 6.713 = 1.067. The authors must correct the table or rerun the experiments; as printed, neither the abstract's 'notable improvement' nor the conclusion's summary is supported by the data.","section":"§3, Table 2; §4 (Conclusion)"},{"comment":"The evaluation protocol assigns a single participant-level PHQ-8 score to every five-utterance clip from that participant and then averages clip-level predictions back to a participant score. Because all clips from one participant share the same target, the model could in principle learn to recognize participant identity or interview-level acoustic properties rather than depression-relevant content within each clip. The manuscript provides no analysis of within-participant label homogeneity, no participant-wise cross-validation, and no baseline that controls for participant identity. This is load-bearing because the headline MAE/RMSE values depend on this clip-to-participant aggregation, and the reported numbers therefore cannot be interpreted as evidence of depression detection per segment.","section":"§2.2.2 (Data Preprocessing)"},{"comment":"The quality check for knowledge injection is self-referential: PsycholexLLAMA generates the fifty questions, and the same model then scores answers from the original and trained models, with a pass@2-style maximum taken. There is no human evaluation or external benchmark. Because the trained model was fine-tuned on the same style of DeepSeek-generated Q&A pairs, its higher average score (8.20 vs. 7.32) may reflect stylistic conformity or in-domain self-preference rather than improved psychological knowledge. This check does not by itself validate the claim that the injected knowledge improves depression scoring.","section":"§3, knowledge-injection quality check"}],"minor_comments":[{"comment":"The text says the authors design 'six distinct types' of questions, but the enumerated list contains only five items: definition, rationale, symptoms, extended knowledge, and critical thinking. The count should be corrected or the sixth type should be stated explicitly.","section":"§2.3.1 (Psychology Knowledge Injection)"},{"comment":"The claim of being 'the first application of LLMs to multimodal depression detection using the DAIC-WOZ dataset' is stated too strongly. The cited related work by Sadeghi et al. (2024) uses an LLM combined with facial-expression features for PHQ-8 regression, and Tank et al. (2024) applies LLMs to textual and audio-visual modalities; the novelty claim needs qualification in light of these and similar prior works.","section":"Abstract and §1"},{"comment":"Equation (3) writes 'PHQ-8 = Linear(LLaMA(Embaudio)−1)', but the superscript '−1' is unexplained and appears to be a typographical artifact. Please clarify whether it denotes the last hidden layer or something else.","section":"§2.3.2, Eq. (3)"},{"comment":"There are several minor presentation issues: the duplicated sentence about lexicon-based sentiment indicators in §2.1.1, irregular spacing in Table 2 row labels such as 'Ours(audio +text+ Knowledge Injection)', and the vague phrase 'mapped it to text-based LLMs' in the abstract. These should be cleaned up in a revision.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The stress-test concern lands: the key numbers in Table 2 are internally inconsistent with the paper's central claims, and the evaluation protocol has a participant-label-repetition problem that is not addressed. Even if the table were a typographical error, the methodological issues would require substantial rework, and the current manuscript does not provide a reliable basis for the stated conclusions. I would encourage the authors to re-analyze the data with a participant-disjoint evaluation and to report whether the audio, text, and knowledge-injection components each contribute under that protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Fair warning before you read: the paper is a cleanly written application of known components, with code and a fixed seed, but the headline claim does not survive Table 2. The full \"audio+text+knowledge injection\" model gets MAE 5.356 / RMSE 6.713, which is exactly the same rounded numbers as both \"audio+text\" and \"audio+knowledge injection\". Adding text to audio and adding knowledge to the full model changes nothing. The comparison that supports \"notable improvement\" is against the AVEC2016 audio baseline (5.72/7.78), while the same table lists an LSTM with RMSE 6.59, better than the proposed model's 6.713. That is an internal contradiction, not a matter of taste.\n\nWhat is genuinely useful: the paper reports ablations, sets a seed, links code, and describes the pipeline (Wav2Vec 2.0, a projector into LLaMA, LoRA, and DeepSeek-generated WHO Q&A pairs). The text-only gain from knowledge injection is real within the paper: MAE drops from 6.342 to 5.354 and RMSE from 8.891 to 6.429. That is the one result I would take seriously, if it were evaluated non-circularly.\n\nThe soft spots are serious. First, the knowledge-injection evaluation is a self-loop: PsycholexLLaMA generates the fifty questions, scores the answers, and no human or independent benchmark is involved. The 7.32 vs 8.20 could reflect grader preference. Second, every clip from a participant carries the same PHQ-8 label; the paper averages clip predictions to person level, but never checks whether segments are homogeneous or whether the model is memorizing speaker identity. Third, the \"first application of LLMs to multimodal depression detection on DAIC-WOZ\" claim is contradicted by the paper's own citations (Sadeghi et al. 2024; Tank et al. 2024). Fourth, only one seed and no uncertainty intervals.\n\nThe code availability and fixed seed are points in its favor, but they don't fix the central empirical problem. If the identical entries in Table 2 are a typographical error, a corrected table might change the picture; as printed, the paper's main claim fails on its own evidence.\n\nRecommendation: I would not send this to peer review in current form. Desk reject, with encouragement to fix the table, replace the circular knowledge evaluation, and rerun the ablations. If the corrected numbers actually show a multimodal or knowledge-injection gain, the paper could become a modest workshop contribution.","headline":"Table 2 as printed contradicts the paper's headline: the full multimodal pipeline is identical to audio-only on MAE/RMSE, so the claimed benefit isn't there.","tokens_in":12117,"tokens_out":2686,"would_cite":false,"duration_ms":24990,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that an LLM pipeline fusing Wav2Vec audio embeddings with transcripts and post-trained on psychology Q&A pairs estimates PHQ-8 depression severity on DAIC-WOZ more accurately than the AVEC2016 audio baseline.","keywords":["depression recognition","large language models","multimodal fusion","psychological knowledge injection","DAIC-WOZ","PHQ-8","Wav2Vec 2.0","speech-based LLM"],"falsifier":"Compute the within-participant variance of the model's clip-level PHQ-8 predictions: if predictions are almost constant inside a participant and vary mainly across participants, that is direct evidence the reported MAE/RMSE comes from person-level label repetition rather than from detecting depression in each clip.","tokens_in":11043,"feed_emoji":"🧠","tokens_out":8741,"duration_ms":87537,"temperature":0.7,"pith_summary":"The paper claims that a large language model can be turned into a spoken-language depression severity estimator by combining audio and text. It projects Wav2Vec 2.0 audio features into LLaMA's embedding space, adds interview transcripts, and first post-trains the model on thousands of question-answer pairs distilled from clinical classification entries. On the DAIC-WOZ test set the complete pipeline reaches MAE 5.356 and RMSE 6.713, below the AVEC2016 audio baseline's 5.72 and 7.78. The authors also report that the psychology-knowledge injection improves the text-only model from MAE 6.342 to 5.354 and RMSE 8.891 to 6.429. A sympathetic reader would take the paper as evidence that knowledge-injected LLMs are a viable route for multimodal mental-health assessment, not as a deployed diagnostic.","feed_headline":"Audio-text LLM beats depression-score baseline with psychology Q&A","feed_subtitle":"A wav2vec-to-LLaMA pipeline post-trained on psychology Q&A pairs lowers MAE and RMSE on DAIC-WOZ interviews.","key_machinery":"The load-bearing machinery is a two-stage fine-tuning protocol on LLaMA. First, a knowledge-injection stage trains the model with supervised learning to answer 4,920 DeepSeek-generated question-answer pairs about depression drawn from clinical classification entries, using question types such as definition, rationale, symptoms, extended knowledge, and critical thinking. Second, a multi-stream stage feeds Wav2Vec 2.0 embeddings through a feed-forward projector to align them with LLaMA's text embedding space, trains the model on transcript prompts of the form 'Transcripts:..., PHQ Score:' and on projected audio, then applies a linear head to the last hidden state to output a PHQ-8 score; LoRA keeps the fine-tuning cheap. This two-stage design is what lets a text-native LLM consume acoustic cues and domain knowledge at once.","core_discovery":"The central discovery claimed is that a two-stage pipeline—Wav2Vec 2.0 for audio, a feed-forward projector, LLaMA as backbone, and a supervised post-training stage on 4,920 Q&A pairs generated from clinical depression entries—regresses PHQ-8 scores at the segment level and averages segment predictions per participant, beating the AVEC2016 audio baseline by about 0.36 MAE and 1.07 RMSE. The comparison includes ablations: audio-only, text-only, audio+text, each with and without knowledge injection. According to the paper, knowledge injection helps the text stream strongly and helps the audio stream slightly, while adding audio to text plus knowledge does not substantially change the numbers further.","pith_inferences":["Beyond the paper, the same knowledge-injection protocol could be tested on per-utterance or per-visit depression labels to see whether the model captures within-interview symptom dynamics rather than only person-level severity.","A direct extension would be to feed the model's clip-level predictions into a clinical decision-support display: if the averaging step hides high-variance segments, flagging segments with extreme predicted scores might give clinicians more useful signals than the averaged participant number.","One consequence the authors do not develop is that their Q&A-generation step could be repeated for other disorders in the same classification source, turning the pipeline into a general knowledge-injected health LLM rather than a depression-specific one.","Because the knowledge injection is text-only, an audio-description variant—pairing each clip with a text caption of its prosody, pauses, and pitch variability—would test whether the transfer from text knowledge to audio can be strengthened, which the authors list as future work."],"forward_implications":["If the result holds, a text-native LLM can serve as the fusion backbone for acoustic and lexical depression cues without a dedicated speech-language model.","Injecting structured psychological knowledge before task fine-tuning is a transferable way to improve LLM-based mental-health scoring, with the larger effect on text than on audio.","The reported margin over the AVEC2016 audio baseline is 0.364 MAE and 1.067 RMSE for the full model, so the practical value lies in the pipeline design rather than in a large accuracy jump.","The near-equal scores for audio+text+knowledge, audio+knowledge, and audio+text suggest that with LoRA-scale tuning the audio stream dominates or saturates the fusion, making larger-capacity training the natural next step named by the authors."],"supporting_citations":[{"why":"Supplies the DAIC-WOZ interview corpus with transcripts, audio, and per-participant PHQ-8 scores that the whole evaluation is built on.","marker":"Gratch et al. (2014)"},{"why":"Provides the AVEC2016 audio baseline (5.72 MAE, 7.78 RMSE) that the paper compares against as the target to beat.","marker":"Valstar et al. (2016)"},{"why":"Provides Wav2Vec 2.0, the self-supervised speech encoder used to extract audio embeddings for the LLM.","marker":"Baevski et al. (2020)"},{"why":"Supplies LLaMA, the base LLM that is post-trained and used as the fusion and regression backbone.","marker":"Touvron et al. (2023)"},{"why":"Supplies the SALMONN-style audio-to-text projection design that the paper adapts, replacing Whisper with Wav2Vec 2.0.","marker":"Tang et al. (2023)"},{"why":"Motivates the question-answer knowledge-injection method and provides the evaluation model used in the pass@2 check.","marker":"Abbasi et al. (2024)"},{"why":"Provides DeepSeek-V3, the LLM used to generate the 4,920 question-answer pairs from clinical entries.","marker":"Liu et al. (2024a)"},{"why":"Provides LoRA, the low-rank adaptation method used to fine-tune LLaMA under limited GPU resources.","marker":"Hu et al. (2022)"},{"why":"Supplies the learning-theory question taxonomy that structures the knowledge-injection prompts.","marker":"Novak and Gowin (1984)"}],"fun_headline_variants":["LLM + audio + psychology Q&A cuts depression score error","First LLM multimodal depression detector hears audio cues","Psychology-informed LLM improves spoken depression detection","LLM with psychology Q&A betters depression MAE and RMSE"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that every five-utterance clip from a participant is equally well described by that participant's single whole-interview depression score, so the model is trained and scored against duplicated participant-level labels without evidence that segments are homogeneous.","fun_headline_variants_meta":{"raw":{"variants":["LLM + audio + psychology Q&A cuts depression score error","First LLM multimodal depression detector hears audio cues","Psychology-informed LLM improves spoken depression detection","LLM with psychology Q&A betters depression MAE and RMSE"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000675,"raw_usage":{"total_tokens":3057,"prompt_tokens":917,"completion_tokens":2140,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":533,"completion_tokens_details":{"reasoning_tokens":2074}},"tokens_in":533,"tokens_out":2140,"duration_ms":19243,"temperature":1.0,"reasoning_tokens":2074,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:58:45.349514+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the within-participant variance of the model's clip-level PHQ-8 predictions: if predictions are almost constant inside a participant and vary mainly across participants, that is direct evidence the reported MAE/RMSE comes from person-level label repetition rather than from detecting depression in each clip.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the DAIC-WOZ interview corpus with transcripts, audio, and per-participant PHQ-8 scores that the whole evaluation is built on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the AVEC2016 audio baseline (5.72 MAE, 7.78 RMSE) that the paper compares against as the target to beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides Wav2Vec 2.0, the self-supervised speech encoder used to extract audio embeddings for the LLM."},{"cited_title":"PsychoLex: Unveiling the Psychological Mind of Large Language Models","cited_arxiv_id":"2408.08848","evidence_quote":"Motivates the question-answer knowledge-injection method and provides the evaluation model used in the pass@2 check."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the learning-theory question taxonomy that structures the knowledge-injection prompts."}],"review_version":1}