{"id":"6499763c-cdc5-4505-a4fc-da65560dcc9c","arxiv_id":"2509.22153","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A speech LLM with a mixture of DoRA experts (MoDE) improves cross-task suicide risk detection accuracy (0.656 vs 0.635 joint tuning) and calibration on 1,223 Chinese adolescents across ten speech tasks, though the unseen-paradigm generalization claim is untested.","lead":"This paper tests a single speech-based AI model that handles ten different speaking tasks to detect suicide risk in teenagers. The proposed mixture-of-experts model beats single-task and simple multi-task baselines in accuracy and confidence, but the authors overstate its ability to handle never-seen tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's claim that MoDE 'can generalise to unseen paradigms' is not tested anywhere: all ten tasks appear in training, so the paper's distinctive generalization claim rests on no experiment.","rationale":"The reader's weakest_assumption focuses on MINI-KID label reliability and test-set refinement, which are real but acknowledged limitations that affect all compared methods roughly equally. The more load-bearing gap is the untested 'unseen paradigms' claim: the paper's title and abstract promise generalization to new elicitation paradigms, yet Section 4 only evaluates the ten tasks that were used in training. This is not an internal inconsistency, but it means the strongest part of the claim is currently unsupported. The correct response is to keep the CONDITIONAL verdict and require a leave-one-task-out experiment before accepting the generalization and calibration claims. I agree with the reader's overall conditional assessment, but I would anchor the condition on the missing held-out-task evaluation rather than on the label-noise concern.","tokens_in":7970,"tokens_out":4206,"duration_ms":42293,"concrete_test":"Run a leave-one-task-out evaluation: for each SSR task t in {01,...,10}, train MoDE and joint tuning on the remaining nine tasks using participant-disjoint train/dev/test splits, then evaluate on task t's test split without any task-specific adaptation or finetuning. Report per-task and average held-out accuracy and ECE, comparing against the majority-class baseline and against the in-domain separate-tuning numbers from Table 2. If held-out accuracy is not clearly above majority class across tasks, or if calibration on held-out tasks is not better than separate tuning, the abstract's 'unseen paradigms' and 'better confidence calibration' claims should be removed or explicitly restricted to in-distribution tasks.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim includes an 'unseen paradigm' capability, but every reported result is in-distribution. Section 3.2 splits participants 8:1:1 and then refines the test set to participants who completed all ten tasks; all ten tasks are present in training and test. Section 4.2/Table 2 thus measures average accuracy on the ten training tasks only. There is no leave-one-task-out or held-out-paradigm experiment, and Section 4.6's routing analysis is descriptive, not predictive. Consequently, the sentence 'Moreover, it can generalise to unseen paradigms' in the abstract is unsupported by the reported evidence. This matters more than the acknowledged MINI-KID label limitation because it is the paper's distinctive contribution: a single model that transfers to new elicitation paradigms. If the model only works on tasks seen during training, the title's 'paradigm-general' framing is not established. A secondary issue reinforces this: Table 4's calibration comparison omits the joint-tuning baseline, so the claimed calibration benefit cannot be attributed to MoDE or even to cross-task learning; but the primary gap is the absent held-out-task evaluation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MoDE, a mixture of DoRA experts on a speech LLM backbone (Qwen2.5-Omni-7B), to unify ten speech-based suicide risk (SSR) assessment tasks into a single model. It reports accuracy comparisons against separate tuning and conventional joint tuning on a dataset of 1,223 Chinese adolescents, ablations of the MoDE components, an analysis of expert-task specialization, and calibration/reject-option experiments. The central claims are that MoDE improves detection accuracy and confidence calibration over both single-task and joint-tuning baselines, and that the model 'can generalise to unseen paradigms.'","tokens_in":8341,"tokens_out":3973,"duration_ms":33835,"significance":"If the accuracy gains are taken at face value, the paper is a useful demonstration that a single speech LLM with lightweight routed adapters can serve multiple suicide-risk elicitation tasks, with potential practical efficiency benefits. The strengths include a relatively large clinically annotated dataset, ten diverse tasks, three-seed reporting with standard errors, and clearly presented tables. However, the paper's most distinctive claim—generalization to unseen paradigms—is not tested at all in the present evaluation, and the calibration comparison omits the joint-tuning control that would be needed to attribute the improvement to cross-task learning. As it stands, the demonstrated scope is substantially narrower than the abstract and conclusions claim.","major_comments":[{"comment":"The claim that MoDE 'can generalise to unseen paradigms' is not supported by any reported experiment. The data split in Section 3.2 is participant-level 8:1:1, so all ten tasks appear in both training and test; no task or paradigm is held out. A leave-one-task-out or similar held-out-paradigm evaluation is required to substantiate the generalization claim, or the claim must be removed from the abstract and conclusions.","section":"Abstract; Section 3.2; Section 4.2"},{"comment":"The calibration comparison is made only between 'Separate' tuning and 'MoDE'. The conventional joint-tuning baseline (Qwen Omni joint tuning, already reported in Section 4.1/Figure 3) is not evaluated for ECE, MCE, NLL, NCE, AUROC, or AUPRC. Therefore the conclusion that 'cross-task learning improves confidence calibration' is not established; the improvement could be due to the MoDE-specific components (router, load balancing, router temperature) rather than joint training. Please add calibration metrics for the joint-tuning baseline.","section":"Section 4.7; Table 4"},{"comment":"The MINI-KID label limitation is acknowledged, but the refinement of the test set to only participants who completed all ten tasks may introduce selection bias. The paper does not report how many participants were excluded, nor does it compare the demographic or clinical characteristics of included versus excluded participants. Because all reported numbers are measured against these labels and on this refined set, please quantify the refinement and discuss its impact on the conclusions, or report results on the full test set.","section":"Section 2.1; Section 3.2; Section 6"},{"comment":"The statement that this is 'the first work that investigates suicide risk detection using speech LLM' appears to conflict with the authors' own reference [5], which uses Whisper and large language models for spontaneous speech-based suicide risk detection, and with reference [6], the SpeechWellness Challenge that incorporates multiple speech tasks. Please clarify the precise novelty relative to these prior works, or soften the claim.","section":"Section 1; References [5], [6]"}],"minor_comments":[{"comment":"The manuscript header reads 'TOW ARDS CROSS-TASK SUICIDE RISK DETECTION VIA SPEECH LLM' while the submitted title is 'Towards Paradigm-General Suicide Risk Detection via Speech LLM'; 'cross-task' and 'cross-paradigm' are used interchangeably throughout, and this terminology should be unified.","section":"Title / Header"},{"comment":"The claim that removing load balancing 'collapses to a single DoRA expert' is not substantiated by any reported quantity; please report expert utilization metrics or router entropy for the ablation without load balancing.","section":"Section 4.3; Table 3"},{"comment":"Temperature scaling is applied to the router softmax rather than to the final classification probabilities; the terminology is potentially confusing because temperature scaling is a standard post-hoc calibration method for output probabilities. Please clarify that this is a router-temperature hyperparameter and justify its role in the training objective.","section":"Section 2.4"},{"comment":"The router architecture (e.g., number and size of hidden layers, which transformer layer provides the hidden states) and the exact DoRA parameterization per expert are not fully specified; additional implementation details would improve reproducibility.","section":"Section 2.3; Section 3.2"},{"comment":"Several figures contain rendering artifacts (e.g., Unicode sequences in captions and axis labels) and low text resolution; please re-export the figures at higher quality.","section":"Figures 3–6"},{"comment":"There are several typographical errors, including 'difference task' (Section 4.1), 'its not only' (Section 4.7), and 'TOW ARDS' in the title; these should be corrected in a final pass.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The manuscript would be strengthened substantially by either adding a leave-one-task-out experiment or removing the 'unseen paradigms' generalization claim. The calibration section also needs the joint-tuning baseline to support the cross-task-learning attribution. The 'first work' claim should be checked against the authors' own prior publications."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read on arXiv:2509.22153. The bottom line: the MoDE accuracy numbers look credible and the cross-task setup is a sensible extension, but the abstract's claim that the model 'can generalise to unseen paradigms' is not tested anywhere. Every experiment is in-distribution: the 8:1:1 split is by participant, the ten tasks all appear in training, and Table 2 reports average accuracy on those same ten tasks. There is no leave-one-task-out or held-out-paradigm run. That sentence should be removed or, better, backed by a real held-out-task experiment.\n\nWhat is new and worth credit: the combination of a speech LLM (Qwen-Omni-7B) with a mixture of DoRA experts for suicide risk detection, evaluated on 1,223 Chinese adolescents across ten speech tasks. The gains are modest but reasonably consistent: average accuracy 0.656 ± 0.003 versus 0.635 ± 0.005 for joint tuning and 0.628 ± 0.006 for separate tuning, with three seeds and standard errors. The decoupled-router design (router uses the backbone without adapters) is a clear, sensible choice. The ablation shows load balancing matters; without it the model collapses to a single expert. The routing heatmaps align with task content, which is a nice sanity check. The authors also state the MINI-KID limitation in Section 6, which is honest.\n\nSoft spots. The calibration comparison in Table 4 only pits separate tuning against MoDE. The joint-tuning baseline is missing, so you cannot attribute the better ECE and rejection curves to cross-task learning. That is a fixable comparison, but as written the claim in Section 4.7 is under-supported. The test-set refinement to participants who completed all ten tasks is a possible selection-bias source; the paper does not report how many participants were dropped or whether the refined cohort still mirrors the full group. Minor: there is a corrupted unicode passage between Figures 5 and 6 that needs cleanup.\n\nThe central accuracy result holds up. The problem is the gap between the 'paradigm-general' framing and the in-distribution evidence. For a conference or workshop paper this is fine after a minor revision; I would not desk-reject it. If you are in speech mental health, it is worth citing for the MoDE design, but do not cite the unseen-paradigm claim until it is actually tested.","headline":"Credible MoDE cross-task accuracy with an unsupported 'unseen paradigms' claim in the abstract.","tokens_in":8744,"tokens_out":3636,"would_cite":true,"duration_ms":30492,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single speech model with routed low-rank experts can detect suicide risk across ten distinct speaking tasks more accurately than task-specific models, and its confidence scores are better calibrated.","keywords":["suicide risk detection","speech LLM","mixture of experts","DoRA","cross-task learning","confidence calibration","adolescent mental health","speech elicitation paradigms"],"falsifier":"Re-run the comparison on the full 1,223-participant cohort without dropping partial completers, and replace self-report MINI-KID labels with clinician-administered structured assessment; if MoDE's average accuracy no longer beats separate tuning or its ECE no longer beats joint tuning, the claimed gains were artefacts of label noise or test-set selection.","tokens_in":7786,"feed_emoji":"🗣️","tokens_out":8449,"duration_ms":66360,"temperature":0.7,"pith_summary":"The paper claims that speech-based suicide risk assessment does not require a separate model for each elicitation task. A single speech large language model, augmented with a mixture of weight-decomposed low-rank experts and a learned router, can jointly handle ten spontaneous-speech tasks—verbal fluency, reading, open-ended answers, face description, and divergent thinking—and outperforms both separate per-task models and ordinary joint training. On 1,223 adolescents, the average accuracy rises from 0.628 to 0.656, a 4.5 per cent relative gain, while expected calibration error drops from 0.099 to 0.061. The authors conclude that unifying paradigms improves detection accuracy, modelling efficiency, and reliability for medical deployment.","feed_headline":"Routed speech experts beat task-specific suicide-risk models","feed_subtitle":"Cross-task training also improves confidence calibration, so uncertain calls can be sent for human review.","key_machinery":"The central object is MoDE, a mixture of DoRA experts grafted onto a speech LLM. DoRA (weight-decomposed low-rank adaptation) splits each pretrained weight into a magnitude scalar and a direction, and applies low-rank updates only to the direction; MoDE keeps ten such adapters and computes a softmax distribution over them from the Transformer's last hidden state. The router is trained on the frozen backbone before the experts are activated, a decoupling intended to keep routing stable. A KL-divergence load-balancing term prevents expert collapse, and temperature scaling on the router's softmax controls routing sparsity. Together these components let a single model act as a task-specific expert on some inputs and a shared generalist on others.","core_discovery":"The central claim is that cross-task training through MoDE improves both accuracy and calibration over single-task models. The paper presents a single Qwen2.5-Omni-7B speech LLM whose transformer weights are updated by a weighted combination of ten DoRA experts, with a router deciding expert weights from the hidden states. In experiments, MoDE reaches 0.656 average accuracy across ten SSR tasks versus 0.628 for separate tuning and 0.635 for joint tuning; the largest task-level gain is on the 'manage distress' question (0.684 to 0.758). The model also records the best ECE (0.061), MCE (0.089), NLL, NCE, AUROC and AUPRC among compared configurations, and its accuracy rises as low-confidence predictions are rejected, while separate tuning's rejection curve is flat. The abstract further asserts that the model can generalise to unseen paradigms, meaning a routing pattern learned from known tasks should transfer to a new elicitation task.","pith_inferences":["The calibration gain may be the most consequential claim for practice: if it holds outside this dataset, cross-task training could become a cheap way to make medical speech classifiers safer without a separate calibration stage.","The routing pattern hints that the router is discovering cognitive dimensions (phonetic reading, emotional face description, divergent thinking) rather than only task identity; one could test this by comparing expert activation matrices with text-only task embeddings.","A direct extension is to treat unseen-paradigm generalisation as a zero-shot transfer experiment: train MoDE on nine tasks, freeze router and experts, evaluate on the tenth, and report accuracy and calibration on that held-out task.","The dataset is roughly balanced, so accuracy is a meaningful headline metric, but real clinics face different base rates; calibration metrics like NLL should be rechecked under prevalence shift before deployment."],"forward_implications":["A deployment pipeline could serve all ten assessment tasks from one model checkpoint, replacing ten separately tuned detectors and simplifying maintenance.","Clinicians can use the model's confidence scores as a triage signal: the rejection analysis shows that withholding low-confidence predictions raises accuracy, so uncertain cases could be routed to human review.","The learned specialisation is inspectable: reading tasks mostly trigger one expert, face-description tasks another, and the divergent-thinking task a unique combination, so routing patterns can be checked per paradigm.","Because manual task-to-expert assignment performs worse than learned routing, forcing a priori grouping of tasks sacrifices useful cross-task cooperation.","If the claimed unseen-paradigm generalisation holds, cross-task training should carry over to new speech-based screening tasks without retraining a fresh model."],"supporting_citations":[{"why":"Supplies the adolescent spontaneous-speech dataset (1,223 participants) and the baseline detection approach this work extends.","marker":"[5]"},{"why":"The SpeechWellness Challenge providing prior evidence that combining multiple SSR tasks helps prediction, motivating cross-task unification.","marker":"[6]"},{"why":"Defines DoRA, the weight-decomposed low-rank adaptation used as the expert adapters.","marker":"[18]"},{"why":"The MINI-KID suicidality module that provides the ground-truth risk labels for evaluation.","marker":"[19]"},{"why":"Temperature scaling, applied to the router softmax and used as the calibration post-processing technique.","marker":"[27]"},{"why":"Qwen2.5-Omni, the speech LLM used as the backbone model.","marker":"[28]"},{"why":"Whisper-large-v3, the speech foundation model baseline.","marker":"[29]"},{"why":"Gives the expected calibration error (ECE) metric used to compare calibration.","marker":"[30]"},{"why":"Gives the maximum calibration error (MCE) metric, also used in the calibration comparison.","marker":"[31]"}],"fun_headline_variants":["Cross-task speech model beats single-task suicide-risk detectors","One speech LLM, ten tasks, better suicide-risk calls","MoDE experts unify speech paradigms for sharper risk detection","Cross-paradigm training improves suicide-risk speech AI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole evaluation leans on the self-report MINI-KID suicidality interview being a trustworthy binary ground truth for current suicide risk, and on the refined test set of participants who finished all ten tasks being representative of the full 1,223-person cohort.","fun_headline_variants_meta":{"raw":{"variants":["Cross-task speech model beats single-task suicide-risk detectors","One speech LLM, ten tasks, better suicide-risk calls","MoDE experts unify speech paradigms for sharper risk detection","Cross-paradigm training improves suicide-risk speech AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000388,"raw_usage":{"total_tokens":2020,"prompt_tokens":889,"completion_tokens":1131,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":505,"completion_tokens_details":{"reasoning_tokens":1065}},"tokens_in":505,"tokens_out":1131,"duration_ms":7667,"temperature":1.0,"reasoning_tokens":1065,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:43:54.994920+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the comparison on the full 1,223-participant cohort without dropping partial completers, and replace self-report MINI-KID labels with clinician-administered structured assessment; if MoDE's average accuracy no longer beats separate tuning or its ECE no longer beats joint tuning, the claimed gains were artefacts of label noise or test-set selection.","supporting_citations":[{"cited_title":"This paper investigates unifying diverse speech tasks within a single model for cross-task suicide risk detection","cited_arxiv_id":null,"evidence_quote":"Supplies the adolescent spontaneous-speech dataset (1,223 participants) and the baseline detection approach this work extends."},{"cited_title":"Though MINI-KID is widely used and often treated as a benchmark for adolescent assessment, it is a self-report tool and cannot fully capture the multifaceted nature of suicide risk","cited_arxiv_id":null,"evidence_quote":"The SpeechWellness Challenge providing prior evidence that combining multiple SSR tasks helps prediction, motivating cross-task unification."},{"cited_title":"Mixture of experts: A literature survey,","cited_arxiv_id":null,"evidence_quote":"The MINI-KID suicidality module that provides the ground-truth risk labels for evaluation."},{"cited_title":"Blowing in the wind: Using ‘North Wind and the Sun’ texts to sample phoneme in- ventories,","cited_arxiv_id":null,"evidence_quote":"Temperature scaling, applied to the router softmax and used as the calibration post-processing technique."},{"cited_title":"Construction and analysis of emotional corpus,","cited_arxiv_id":null,"evidence_quote":"Qwen2.5-Omni, the speech LLM used as the backbone model."},{"cited_title":"Study on Chinese word sentiment po- larity automatic estimation,","cited_arxiv_id":null,"evidence_quote":"Whisper-large-v3, the speech foundation model baseline."},{"cited_title":"The racially diverse affective expression (RADIATE) face stimulus set,","cited_arxiv_id":null,"evidence_quote":"Gives the expected calibration error (ECE) metric used to compare calibration."},{"cited_title":"Divergent thinking as an indicator of creative potential,","cited_arxiv_id":null,"evidence_quote":"Gives the maximum calibration error (MCE) metric, also used in the calibration comparison."}],"review_version":2}