Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

GOAT-SLM: A Spoken Language Model with Paralinguistic and Speaker Characteristic Awareness

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GOAT-SLM claims a spoken language model can track dialect, emotion, age, and coughs, and adjust its spoken replies accordingly.

desk verdict A real architectural contribution to spoken language models, but the headline outperformance claims rest on the authors' own benchmark and synthetic data pipeline, so treat them as provisional pending independent evaluation. read the letter →

arxiv 2507.18119 v2 pith:QMYHCMJS submitted 2025-07-24 cs.CL cs.AIcs.SDeess.AS

Hongjie Chen , Zehan Li , Yaodong Song , Wenming Deng , Yitong Yao , Yuxin Zhang , Hang Lv , Xuechao Zhu
show 8 more authors
Jian Kang Jie Lian Jie Li Chao Wang Shuangyong Song Yongxiang Li Zhongjiang He Xuelong Li
This is my paper · ORCID
classification cs.CLcs.AIcs.SDeess.AS
keywords spokenlanguagemodelparalinguisticawarenessdual-modalityheaddialectfollowingemotion-awarespeechgenerationnon-speechvocalizationspeech-textalignmentage-awaredialogue
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most spoken language models treat speech as a wrapper for text and ignore how something is said. GOAT-SLM tries to fix this by building a spoken dialogue model that tracks dialect, emotion, age, and non-speech sounds like coughing, and lets those cues change both what it replies and how it sounds. The paper's central claim is that its dual-modality head architecture — shared reasoning layers that branch into separate text and speech output heads — plus a three-stage training pipeline makes this paralinguistic awareness possible without sacrificing general knowledge QA. On the authors' TELEVAL benchmark, GOAT-SLM beats all compared open-source models on dialect following, non-speech vocal response, age-appropriate response, and emotional speech generation, while keeping competitive semantic QA scores. A reader should care because it points to a concrete recipe for making voice assistants socially aware rather than merely accurate at transcribing words.

What carries the argument

The load-bearing mechanism is the dual-modality head architecture: the bottom 15 layers of TeleChat2-7B act as a shared 'Think' reasoning core, and the top 15 layers branch into a 'Write' text head and a 'Speak' speech-token head, with the speech head initialized from the text head to transfer knowledge. This design lets linguistic reasoning stay intact while a parallel acoustic pathway learns expressive realization. The second mechanism is the orchestrated three-stage training: attribute-prompted instruction tuning, speech-text alignment via self-distilled repeat-and-continue targets, and attribute-aware refinement where GOAT-TTS converts text responses into stylized speech targets (emotion, listener age, dialect). A confidence-based gradient masking strategy and multi-token prediction support stable, low-latency speech generation. Together these parts carry the claim that acoustic realization can be trained independently of semantic content and then re-coupled through shared hidden representations.

What would settle it

Run GOAT-SLM on an independently collected spoken dialogue test set whose dialect, emotion, age, and non-speech-vocal labels come from human speakers and human raters, with no GOAT-TTS synthesis involved; if the model's dialect-following and non-speech-vocal margins over Qwen2.5-Omni and Kimi-Audio shrink to near zero, the central claim of autonomous paralinguistic awareness is not sustained.

Watch

Extended reading notes

Core claim

The paper claims that a spoken language model can perceive non-linguistic vocal cues and autonomously adapt its responses to them, with no explicit instruction, by separating 'what to say' from 'how to say it'. Concretely, GOAT-SLM keeps the bottom layers of a pretrained LLM as a shared semantic core and splits the top layers into a text-generation head and a speech-token-generation head initialized from the text head. A staged training schedule first teaches the model to respond to attribute-annotated text instructions (emotion, age, dialect, non-speech events), then aligns speech input to text targets through repeat-and-continue self-distillation, and finally refines expressive speech generation using targets synthesized by the same team's text-to-speech system. The reported results on TELEVAL show large margins over open-source baselines: about 51 percent average dialect following versus 19 percent for the next-best model, 41 percent versus 9 percent on non-speech vocal response, 72 percent versus 43 percent on age-aware interaction, and the lowest character error rate with the highest emotion score in spoken response generation. The authors interpret this as evidence that paralinguistic awareness is compatible with, and separable from, linguistic intelligence.

Load-bearing premise

The evaluation rests on TELEVAL, a benchmark built by the same team and partly generated with the team's own text-to-speech system, so the large margins over open-source models could reflect fit to the team's data-generation conventions rather than general paralinguistic ability.

Editorial extensions

If this is right

  • Without changing the underlying LLM, the same dual-head recipe can add dialect, emotion, or age sensitivity to any existing spoken dialogue system.
  • Paralinguistic cues can be carried through speech-token generation even when the text head does not mark the dialect, so the acoustic channel can express what the text channel does not.
  • Models trained this way can respond to non-speech sounds (coughs, laughter, sighs) with socially appropriate verbal replies, not just descriptions of the sound.
  • Because the speech head is initialized from the text head, high text QA capability is preserved: GOAT-SLM remains within a few points of the strongest open-source models on general knowledge AQA while gaining paralinguistic skills.
  • The full recipe is reproducible from open components: Whisper-small encoder, a 7B LLM backbone, and the team's TTS for target synthesis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If TELEVAL's measurements survive independent replication, the attribute-prompted instruction-tuning step is the likely active ingredient; a controlled ablation that removes Stage 1 would test whether the benchmark gains come mostly from text-level awareness rather than from speech-level perception.
  • The high dialect-consistency scores in subjective listening (over 90 percent for four of five dialects) suggest the model learns dialect cues in the acoustic representation itself; this raises the possibility of dialect transfer to languages that have audio data but little transcribed text, a direction the paper does not explore.
  • The authors' own benchmark, built with their own TTS and prompts, may reward pipeline-specific conventions, so the margin over open-source models should be re-checked on externally collected human speech; this is the key open question for adoption.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces GOAT-SLM, an end-to-end spoken language model with a dual-modality head architecture that shares the lower layers of a pretrained LLM as a semantic core and branches into separate text and speech generation heads. Training is staged: instruction tuning with paralinguistic attribute prompts, speech-text alignment on ASR/TTS data with self-distilled targets, and high-fidelity expressive speech generation using synthetic targets produced with the authors' GOAT-TTS system. The model is evaluated on TELEVAL, a benchmark authored by the same group, across semantic tasks (AQA, dialect AQA, multi-turn dialogue) and paralinguistic tasks (dialect following, emotion, non-speech vocal signals, age-aware responses). The central claim is that GOAT-SLM outperforms existing open-source models in emotion, dialectal variation, and age-sensitive interactions while preserving semantic competence.

Significance. If the claims are correct, the paper offers a useful recipe: decoupling linguistic reasoning from acoustic realization via shared LLM layers, and aligning paralinguistic attributes through staged, attribute-aware training. The architecture and training details are described with unusual specificity, including layer counts, data sizes, learning rates, GPU hours, and mechanisms such as multi-token prediction and confidence-based gradient masking, which would help reproducibility. However, the headline comparative claim rests almost entirely on a self-authored benchmark whose test stimuli may share generation conventions with the model's own training data, and the evaluation lacks statistical testing and baseline inclusion in the subjective dialect test. The significance of the contribution is therefore conditional on external validation.

major comments (4)
  1. [Section 5.2, Tables 7 and 8; Section 4.3 (Stage 3-2)] The central claim of outperforming open models in emotion, dialect, and age is supported only by TELEVAL (ref [22]), which is authored by the same team, and the training data in Stage 2-2 and Stage 3-2 are constructed with the same prompted attribute descriptions (Figure 2) and the same GOAT-TTS flow module (ref [21]) that appear to be used to produce the evaluation stimuli in Tables 7 and 8. The large margins (e.g., 50.73% vs. 18.91% in dialect following, 40.91% vs. 9.19% in Para_mix300-zh, 72.13% vs. 42.51% in Age-zh) may therefore reflect the model's fit to the authors' data-generation conventions rather than genuine paralinguistic awareness. Please provide an independent evaluation, or at minimum a leakage analysis demonstrating that the benchmark test clips are not produced with the same pipeline and that the reported advantages persist under different dialect, age, and NSV renditions.
  2. [Section 5.2, Table 10] The subjective dialect evaluation excludes all baseline models because they 'lack dialect follow capabilities'; this makes the 90%+ consistency rates uninterpretable as evidence of superiority and is circular as a justification for exclusion. To support the comparative claim, baseline-generated samples should be included in a blind listening test, or the evaluation should be presented as an absolute capability measure with appropriate caveats. The number of raters per dialect is given (10), but inter-annotator agreement is not reported.
  3. [Section 5.2, Table 8; Abstract] The abstract's claim that GOAT-SLM outperforms existing open-source models in handling emotion is not supported by the ESD-zh result in Table 8, where GOAT-SLM scores 45.31% versus Kimi-Audio's 53.17%. Moreover, all results in Tables 4-8 are point estimates without confidence intervals or significance tests, and some differences are small (e.g., Table 4 LlamaQA-zh 52.67 vs. GLM-4-Voice 53.00; Table 6 84.00 vs. 86.67). Please report variance, significance tests, or at least multiple evaluation runs, and reconcile the emotion claim with the ESD-zh numbers.
  4. [Section 5.2, Table 9] Table 9 reports a metric labeled 'Emotion', but the paper does not define how this metric is computed or who the annotators are. Since the emotion-following claim depends on this metric, the definition, annotation protocol, and reliability (e.g., agreement statistics) must be specified.
minor comments (5)
  1. [Section 2.1] The text cites 'SpeechGPT [11]' but reference [11] is Salmonn-Omni; the correct reference for SpeechGPT appears to be [2]. Please check all citation numbers after the reference list was renumbered.
  2. [Figure 2 caption] The caption contains a typo: 'charateristic' should be 'characteristic'.
  3. [Section 3] The speech encoder is written as 'Whisper-small 1' with a superscript footnote marker; please format the footnote reference consistently, and ensure the model version and source are given in the main text.
  4. [Table 2] It is unclear how 'Hours of Speech' relates to 'Num. Samples' (e.g., 73M samples with 170k hours versus 53M samples with 85k hours); please clarify the average utterance duration and whether these counts refer to spoken or textual samples.
  5. [Section 5.1, Table 4] The statement that GOAT-SLM's general AQA capability 'slightly declines' is understated; large drops occur on TriviaQA-en (37.51 vs. 45.52 for Kimi-Audio) and LlamaQA-zh (52.67 vs. 65.33 for Kimi-Audio). Please discuss this trade-off more explicitly.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline paralinguistic advantage rests on a self-authored benchmark (TELEVAL), with no independent non-semantic evaluation; the claimed outperformance is not yet externally established.

  1. self citation load bearing [Abstract and Section 5 (evaluation setup), with Reference [22]]
    "Experimental results on TELEVAL, a multi-dimensional evaluation benchmark, demonstrate that GOAT-SLM achieves well-balanced performance across both semantic and non-semantic tasks, and outperforms existing open-source models in handling emotion, dialectal variation, and age-sensitive interactions."

    The sole evidence for the headline paralinguistic outperformance is TELEVAL (ref. [22]), whose author list (Z. Li, H. Chen, Y. Zhang, J. Zhou, X. Wang, H. Lv, J. Kang, J. Li, Y. Li, X. Li) overlaps with the present paper's authors. No external, independently authored benchmark is used for emotion, dialect following, NSV, or age-aware interaction. The claim 'outperforms existing open-source models' therefore reduces to performance on the team's own benchmark, making the self-citation load-bearing for the paper's central contribution. External semantic benchmarks (LlamaQA, TriviaQA, etc.) do not independently support the non-semantic claims.

full rationale

The architecture and training pipeline are described in concrete detail, and the semantic AQA results are compared on external datasets (LlamaQA, TriviaQA, WebQ, ChineseSimpleQA, ChineseQuiz) against open-source baselines; those parts are not circular. However, the paper's central non-semantic claims—emotion, dialect following, NSV response, and age-aware interaction—are evaluated only on TELEVAL [22], a benchmark authored by the same team. The claim 'outperforms existing open-source models in handling emotion, dialectal variation, and age-sensitive interactions' therefore rests on a self-citation that is load-bearing. Additionally, Stage 2-2 and Stage 3-2 generate training targets with the same lab's LLM prompting and GOAT-TTS (ref [21]), and the TELEVAL age/NSV test uses synthetic voices/signals; the paper does not state whether those test clips share GOAT-TTS generation conventions, so the near-saturation scores (e.g., Age-zh 72.13% vs. 42.51%) could reflect distribution fit rather than general paralinguistic ability. This is a material unverified link, not an exhibited equation-level reduction. Weighing these, the circularity score is 6: partial, because the self-authored benchmark is the sole support for the headline claim, but the paper does include external semantic benchmarks and transparent baseline comparisons that keep it from being fully self-referential.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard transfer-learning assumptions about pretrained encoders and LLMs, plus the assumption that a self-authored benchmark and self-generated synthetic speech are valid evaluation instruments. No free parameters in the derivation sense are fitted; the training hyperparameters are conventional. No new physical entities are introduced.

assumptions (4)
  • domain assumption Pretrained LLM (TeleChat2-7B) retains its reasoning ability when the bottom 15 layers are used as a shared semantic core and the top layers are split into text and speech heads.
    The whole dual-modality head design and the claim of preserved text intelligence rest on this transfer assumption, which is not proven in the paper.
  • domain assumption Paralinguistic attributes (emotion, dialect, age, non-speech vocalizations) can be captured by Whisper-small features and conditioned through text prompts and speech prompts.
    Stage 1 and Stage 2 training inject attribute descriptions into prompts, assuming these descriptions align with acoustic cues and transfer to audio-only inference such as the ESD-zh evaluation.
  • ad hoc to paper TELEVAL benchmark scores are a valid measure of real-world paralinguistic awareness.
    TELEVAL is authored by the same team (ref [22]), so the benchmark's construct validity is assumed rather than independently established.
  • ad hoc to paper Synthetic speech targets generated by GOAT-TTS are adequate for training and evaluating expressive speech generation.
    Stage 3-2 uses GOAT-TTS to convert text responses into speech targets; if GOAT-TTS has systematic style biases, the model learns those biases.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GOAT-SLM: A Spoken Language Model with Paralinguistic and Speaker Characteristic Awareness." pith.science (2026). https://pith.science/paper/QMYHCMJS

@misc{pith2026250718119,
  author       = {Pith},
  title        = {Pith review of: GOAT-SLM: A Spoken Language Model with Paralinguistic and Speaker Characteristic Awareness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QMYHCMJS}},
  note         = {Machine review of arXiv:2507.18119}
}
read the original abstract

Recent advances in end-to-end spoken language models (SLMs) have significantly improved the ability of AI systems to engage in natural spoken interactions. However, most existing models treat speech merely as a vehicle for linguistic content, often overlooking the rich paralinguistic and speaker characteristic cues embedded in human speech, such as dialect, age, emotion, and non-speech vocalizations. In this work, we introduce GOAT-SLM, a novel spoken language model with paralinguistic and speaker characteristic awareness, designed to extend spoken language modeling beyond text semantics. GOAT-SLM adopts a dual-modality head architecture that decouples linguistic modeling from acoustic realization, enabling robust language understanding while supporting expressive and adaptive speech generation. To enhance model efficiency and versatility, we propose a modular, staged training strategy that progressively aligns linguistic, paralinguistic, and speaker characteristic information using large-scale speech-text corpora. Experimental results on TELEVAL, a multi-dimensional evaluation benchmark, demonstrate that GOAT-SLM achieves well-balanced performance across both semantic and non-semantic tasks, and outperforms existing open-source models in handling emotion, dialectal variation, and age-sensitive interactions. This work highlights the importance of modeling beyond linguistic content and advances the development of more natural, adaptive, and socially aware spoken language systems.

Figures

Figures reproduced from arXiv: 2507.18119 by the authors.

Figure 1
Figure 1. Framework and staged training process. Stage 1: Instruction tuning. Stage 2: speech-text [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Prompts used to generate textual response during the construction of speech–text data pairs. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OSUM-EChat: Enhancing End-to-End Empathetic Spoken Chatbot via Understanding-Driven Spoken Dialogue

    cs.SD 2025-08 conditional novelty 6.0 of 10

    A three-stage trained speech-to-speech chatbot with an explicit think step transfers paralinguistic understanding into empathetic responses, outperforming prior end-to-end spoken dialogue systems on a new LLM-scored b...

  2. BridgeTA: Bridging the Representation Gap in Knowledge Distillation via Teacher Assistant for Bird's Eye View Map Segmentation

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    A teacher-assistant distillation framework bridges the representation gap between LiDAR-camera and camera-only BEV segmentation, improving camera-only mIoU by 4.2% on nuScenes.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages · cited by 2 Pith papers

  1. [22]

    TELEV AL: A dynamic benchmark designed for spoken language models in chinese interactive scenarios,

    Z. Li, H. Chen, Y . Zhang, J. Zhou, X. Wang, H. Lv, J. Kang, J. Li, Y . Li, and X. Li, “TELEV AL: A dynamic benchmark designed for spoken language models in chinese interactive scenarios,” arXiv Preprint, 2025

    work page 2025

  2. [21]

    GOAT- TTS: llm-based text-to-speech generation optimized via A dual-branch architecture,

    Y . Song, H. Chen, J. Lian, Y . Zhang, G. Xia, Z. Li, G. Zhao, J. Kang, Y . Li, and J. Li, “GOAT- TTS: llm-based text-to-speech generation optimized via A dual-branch architecture,” arXiv Preprint, 2025

    work page 2025

  3. [1]

    On the landscape of spoken language models: A comprehensive survey,

    S. Arora, K. Chang, C. Chien, Y . Peng, H. Wu, Y . Adi, E. Dupoux, H. Lee, K. Livescu, and S. Watanabe, “On the landscape of spoken language models: A comprehensive survey,”arXiv Preprint, 2025

    work page 2025

  4. [2]

    Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities,

    D. Zhang, S. Li, X. Zhang, J. Zhan, P. Wang, Y . Zhou, and X. Qiu, “Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities,” in Proc. EMNLP, 2023, pp. 15 757–15 773

    work page 2023

  5. [3]

    Moshi: a speech-text foundation model for real-time dialogue,

    A. Défossez, L. Mazaré, M. Orsini, A. Royer, P. Pérez, H. Jégou, E. Grave, and N. Zeghidour, “Moshi: a speech-text foundation model for real-time dialogue,” arXiv Preprint, 2024

    work page 2024

  6. [4]

    Speechgpt 2.0-preview,

    Open-Moss, “Speechgpt 2.0-preview,” https://github.com/OpenMOSS/SpeechGPT-2.0-preview, 2025

    work page 2025

  7. [5]

    Llama-omni: Seamless speech interaction with large language models,

    Q. Fang, S. Guo, Y . Zhou, Z. Ma, S. Zhang, and Y . Feng, “Llama-omni: Seamless speech interaction with large language models,” in Proc. ICLR, 2025, pp. 57 607–57 624

    work page 2025

  8. [6]

    Freeze-omni: A smart and low latency speech-to-speech dialogue model with frozen LLM,

    X. Wang, Y . Li, C. Fu, Y . Shen, L. Xie, K. Li, X. Sun, and L. Ma, “Freeze-omni: A smart and low latency speech-to-speech dialogue model with frozen LLM,” arXiv Preprint, 2024

    work page 2024

Show all 29 references
  1. [7]

    Slam-omni: Timbre-controllable voice interaction system with single-stage training,

    W. Chen, Z. Ma, R. Yan, Y . Liang, X. Li, R. Xu, Z. Niu, Y . Zhu, Y . Yang, Z. Liu, K. Yu, Y . Hu, J. Li, Y . Lu, S. Liu, and X. Chen, “Slam-omni: Timbre-controllable voice interaction system with single-stage training,” arXiv Preprint, 2024

    2024

  2. [8]

    Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot,

    A. Zeng, Z. Du, M. Liu, K. Wang, S. Jiang, L. Zhao, Y . Dong, and J. Tang, “Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot,”arXiv Preprint, 2024

    2024

  3. [9]

    Minicpm-o 2.6: A gpt-4o level mllm for vision, speech, and multimodal live streaming on your phone,

    MiniCPM-o Team, OpenBMB, “Minicpm-o 2.6: A gpt-4o level mllm for vision, speech, and multimodal live streaming on your phone,” https://github.com/OpenBMB/MiniCPM-o, 2025

    2025

  4. [10]

    Baichuan-omni-1.5 technical report,

    Baichuan Inc., “Baichuan-omni-1.5 technical report,” arXiv Preprint, 2025

    2025

  5. [11]

    Salmonn-omni: A codec-free LLM for full-duplex speech understanding and generation,

    W. Yu, S. Wang, X. Yang, X. Chen, X. Tian, J. Zhang, G. Sun, L. Lu, Y . Wang, and C. Zhang, “Salmonn-omni: A codec-free LLM for full-duplex speech understanding and generation,”arXiv Preprint, 2024

    2024

  6. [12]

    Minmo: A multimodal large language model for seamless voice interaction,

    Q. Chen, Y . Chen, Y . Chen, M. Chen, Y . Chen, C. Deng, Z. Du, R. Gao, C. Gao, Z. Gao, Y . Li, X. Lv, J. Liu, H. Luo, B. Ma, C. Ni, X. Shi, J. Tang, H. Wang, H. Wang, W. Wang, Y . Wang, Y . Xu, F. Yu, Z. Yan, Y . Yang, B. Yang, X. Yang, G. Yang, T. Zhao, Q. Zhang, S. Zhang, N...

    2025

  7. [13]

    Qwen2.5-omni technical report,

    Qwen Team, “Qwen2.5-omni technical report,” arXiv Preprint, 2025

    2025

  8. [14]

    Step-audio: Unified understanding and generation in intelligent speech interaction,

    Step-Audio Team, “Step-audio: Unified understanding and generation in intelligent speech interaction,” arXiv Preprint, 2025. 10

    2025

  9. [15]

    Step-Audio-AQAA: a fully end-to-end expressive large audio language model,

    ——, “Step-Audio-AQAA: a fully end-to-end expressive large audio language model,”arXiv Preprint, 2025

    2025

  10. [16]

    Llama-omni2: Llm-based real-time spoken chatbot with autoregressive streaming speech synthesis,

    Q. Fang, Y . Zhou, S. Guo, S. Zhang, and Y . Feng, “Llama-omni2: Llm-based real-time spoken chatbot with autoregressive streaming speech synthesis,” arXiv Preprint, 2025

    2025

  11. [17]

    Deeptalk: Towards seamless and smart speech interaction with adaptive modality-specific moe,

    H. Shao, H. Gao, Y . Shen, J. Chen, L. Li, Z. Long, B. Tong, K. Li, and X. Sun, “Deeptalk: Towards seamless and smart speech interaction with adaptive modality-specific moe,” arXiv Preprint, 2025

    2025

  12. [18]

    BoSS: Beyond-semantic speech,

    Q. Wang, Z. Li, H. Lv, H. Chen, Y . Song, J. Kang, J. Lian, J. Li, Y . Li, Z. He, and X. Li, “BoSS: Beyond-semantic speech,” arXiv Preprint, 2025

    2025

  13. [19]

    V oila: V oice-language foundation models for real-time autonomous interaction and voice roleplay,

    Y . Shi, Y . Shu, S. Dong, G. Liu, J. Sesay, J. Li, and Z. Hu, “V oila: V oice-language foundation models for real-time autonomous interaction and voice roleplay,”arXiv Preprint, 2025

    2025

  14. [20]

    Kimi-audio technical report,

    K. Team, “Kimi-audio technical report,” arXiv Preprint, 2025

    2025

  15. [23]

    Qwen2-audio technical report,

    Y . Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y . Leng, Y . Lv, J. He, J. Lin, C. Zhou, and J. Zhou, “Qwen2-audio technical report,” arXiv Preprint, 2024

    2024

  16. [24]

    Baichuan-Audio: A unified framework for end-to-end speech interaction,

    T. Li, J. Liu, T. Zhang, Y . Fang, D. Pan, M. Wang, Z. Liang, Z. Li, M. Lin, G. Dong, J. Xu, H. Sun, Z. Zhou, and W. Chen, “Baichuan-Audio: A unified framework for end-to-end speech interaction,” arXiv Preprint, 2025

    2025

  17. [25]

    Telechat technical report,

    Z. Wang, X. Liu, S. Liu, Y . Yao, Y . Huang, Z. He, X. Li, Y . Li, Z. Che, Z. Zhang, Y . Wang, X. Wang, L. Pu, H. Xu, R. Fang, Y . Zhao, J. Zhang, X. Huang, Z. Lu, J. Peng, W. Zheng, S. Wang, B. Yang, X. he, Z. Jiang, Q. Xie, Y . Zhang, Z. Li, L. Shi, W. Fu, Y . Zhang, Z. Huan...

    2024

  18. [26]

    Audiochatllama: Towards general-purpose speech abilities for llms,

    Y . Fathullah, C. Wu, E. Lakomkin, K. Li, J. Jia, Y . Shangguan, J. Mahadeokar, O. Kalinli, C. Fuegen, and M. Seltzer, “Audiochatllama: Towards general-purpose speech abilities for llms,” in Proc. NAACL, 2024, pp. 5522–5532

    2024

  19. [27]

    BLSP: bootstrapping language-speech pre-training via behavior alignment of continuation writing,

    C. Wang, M. Liao, Z. Huang, J. Lu, J. Wu, Y . Liu, C. Zong, and J. Zhang, “BLSP: bootstrapping language-speech pre-training via behavior alignment of continuation writing,” arXiv Preprint, 2023

    2023

  20. [28]

    Wav2prompt: End-to-end speech prompt learning and task-based fine-tuning for text-based llms,

    K. Deng, G. Sun, and P. C. Woodland, “Wav2prompt: End-to-end speech prompt learning and task-based fine-tuning for text-based llms,” in Proc. NAACL, 2025, pp. 6940–6956

    2025

  21. [29]

    DeSTA2: Developing instruction-following speech language model without speech instruction-tuning data,

    K.-H. Lu, Z. Chen, S.-W. Fu, C.-H. H. Yang, J. Balam, B. Ginsburg, Y .-C. F. Wang, and H. yi Lee, “DeSTA2: Developing instruction-following speech language model without speech instruction-tuning data,” arXiv Preprint, 2025. 11

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.