Pith. sign in

REVIEW 3 major objections 3 minor 57 references

Speech Discrete Tokens or Continuous Features? A Comparative Analysis for Spoken Language Understanding in SpeechLLMs

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Under matched training settings, continuous self-supervised speech features generally outperform discrete tokens across six spoken-language-understanding tasks in large language models, with task-dependent efficiency and robustness trade-of

desk verdict Useful abstract on a representation choice for SpeechLLMs, but the body is unreadable and the key fairness claim is unverifiable. read the letter →

arxiv 2508.17863 v1 pith:EBKRLX26 submitted 2025-08-25 cs.CL cs.SD

Dingdong Wang , Junan Li , Mingyu Cui , Dongchao Yang , Xueyuan Chen , Helen Meng This is my paper
classification cs.CLcs.SD
keywords spokenlanguageunderstandingspeechlargemodelsdiscretetokenscontinuousfeaturesself-supervisedlearningcomparativeevaluationrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to settle a practical question: when building a spoken-language-understanding system on top of a speech large language model, should the audio be fed in as discrete tokens or as dense continuous features? The authors claim that, under the same experimental settings, continuous features generally win across six SLU-related tasks and two model scales. They also show that the two input styles behave differently in terms of efficiency, layer usage, and robustness. If the finding holds, it gives practitioners a data-backed default choice when designing SpeechLLMs.

What carries the argument

The controlled comparison protocol is the load-bearing mechanism: discrete tokens and continuous features are both evaluated under identical experimental settings, with analysis of efficiency, SSL layer usage, LLM layer usage, and robustness. This protocol is what turns the empirical performances into a claim about the input representations themselves rather than about the training setups.

What would settle it

Re-running the identical six-task protocol while explicitly controlling for sequence length, training steps, learning rate, and early stopping, and finding that discrete tokens match or beat continuous features on the aggregate score—or, conversely, running a wider task panel and finding the order flips.

Watch

Extended reading notes

Core claim

The paper claims that continuous features derived from self-supervised speech models are generally better than discrete tokens for spoken language understanding in SpeechLLMs. Under what the authors describe as the same experimental settings, they evaluate both representations across six tasks with a small and a large LLM, and find a consistent performance edge for continuous features. They further analyze how each representation consumes SSL layers and LLM layers, and how each withstands perturbations, concluding that the two paradigms process speech information in distinct ways.

Load-bearing premise

The conclusion depends on the six tested tasks and the matched training conditions being representative; if discrete tokens received shorter sequences, fewer steps, or worse hyperparameters in the comparison, the headline result would not follow.

Editorial extensions

If this is right

  • Continuous self-supervised features should be the stronger default input for spoken-language-understanding tasks in SpeechLLMs under comparable budgets.
  • The advantage is not absolute: efficiency and robustness differences mean discrete tokens may still be preferable when compute, sequence length, or noise conditions dominate.
  • SSL layer selection should be tuned separately for each input style, since the two representations use encoder layers differently.
  • The six-task evaluation provides a reusable protocol for comparing speech representations in SpeechLLMs.
  • If the trend generalizes, future SpeechLLM designs could de-prioritize quantization pipelines and feed dense features directly to the language model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The continuous-feature advantage may stem from information density: quantization discards fine phonetic detail that downstream LLMs can still exploit, predicting larger gaps on fine-grained SLU tasks and smaller gaps on coarse ones—testable with a task-difficulty breakdown.
  • A hybrid input—discrete tokens for structure plus a continuous residual for detail—might combine the efficiency of token sequences with the fidelity of dense features; the paper's complementary layer patterns hint at this.
  • Whether the trend extends beyond SLU to ASR, speech translation, or audio question answering is untested; replicating the protocol on those tasks would bound the scope of the conclusion.
  • The paper's 'same experimental settings' claim depends on matching sequence length and training steps between token and frame representations; a reasonable extension is to verify whether the gap persists when sequence lengths are explicitly equalized.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. This manuscript reports a comparative study of SSL-based discrete tokens versus continuous features as input representations for SpeechLLMs. The authors claim to evaluate both paradigms 'under the same experimental settings' across six spoken-language-understanding tasks, using two LLM scales (Qwen1.5-0.5B and Llama3.1-8B), and to conduct analyses of efficiency, SSL-layer usage, LLM-layer usage, and robustness. The headline finding is that continuous features generally outperform discrete tokens. The supplied full text is largely unreadable due to encoding corruption, so I can only assess the abstract and a few fragments; the experimental details cannot be audited from the material provided.

Significance. The question addressed is timely and practically important: practitioners need to know whether to feed SpeechLLMs discrete tokens or dense SSL features. If the result holds under genuinely matched conditions, it would provide a useful default recommendation and would challenge the common assumption that discrete tokens are always preferable for LLM-based speech processing. The use of two LLM scales and six tasks is a strength, and the planned analyses of efficiency, layer usage, and robustness could be valuable. However, the current manuscript does not give the reader access to the evidence needed to assess these claims; the significance therefore remains conditional on the missing experimental detail.

major comments (3)
  1. [Abstract and unreadable Experimental Setup/Results] The headline claim—'continuous features generally outperform discrete tokens'—rests entirely on the assertion that both are evaluated 'under the same experimental settings.' The abstract does not report the concrete controls: training steps, learning rate, batch size, number of runs/seeds, early-stopping policy, and crucially the alignment between token streams (typically ~25 Hz) and continuous frames (typically ~50 Hz). The supplied full text is corrupted to the point of unreadability, so these controls cannot be verified. Please provide the exact training configuration and, ideally, a config file or code release. Without this, the comparative conclusion is not established.
  2. [Abstract ('six SLU-related tasks' vs. 'various tasks')] The generalization from 'six spoken language understanding-related tasks' to 'various tasks' requires per-task results with effect sizes and variance. The abstract reports only an aggregate direction. If the continuous-feature advantage holds on some tasks but is reversed on others, the headline overstates the finding. Please include a full results table with per-task metrics for both LLM scales, and report the direction and magnitude for each task.
  3. [Abstract ('in-depth analyses', 'efficient comparison', 'robustness comparison')] The claimed efficiency, SSL-layer, LLM-layer, and robustness analyses are load-bearing parts of the paper's message, but no quantitative results are readable. For efficiency, report measured training/inference cost, sequence-length or frame-rate differences, and memory usage. For robustness, specify the perturbations and report degradation numbers. Without these numbers, the qualitative statement that each method 'exhibits distinct characteristics' is unsupported.
minor comments (3)
  1. [Full text] The PDF/source text supplied for review is heavily encoding-corrupted (mojibake). This is not a scientific criticism, but it prevents any referee from verifying equations, tables, or experimental settings. A clean version is required before the paper can be properly evaluated.
  2. [Abstract] The phrase 'various tasks' is imprecise; the paper actually evaluates six SLU-related tasks. Please name the tasks in the abstract or at least specify their categories.
  3. [Abstract/Results] The abstract gives no effect sizes, confidence intervals, or significance tests. If these are absent from the paper, they should be added; if present, they were unreadable in the supplied copy.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identified: empirical benchmark against external features and standard tasks, with no fitted-parameter feedback or load-bearing self-citation visible.

full rationale

The paper is an empirical comparative benchmark rather than a derivation chain: it evaluates SSL-based discrete tokens versus continuous features across six SLU tasks using Qwen1.5-0.5B and Llama3.1-8B. The central claim that continuous features generally outperform discrete tokens depends on the fairness of the experimental comparison, which is an empirical premise about matched settings, not a definitional or self-referential reduction. No fitted parameter is subsequently renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation to justify the conclusion. The layer-level and robustness analyses are post hoc interpretations of experimental results, not results derived from their own inputs. The supplied full text is largely corrupted, so no specific equation or self-citation chain can be quoted; under the hard rule requiring an exhibited reduction, no circularity step can be claimed. Concerns about whether the settings were truly matched, or whether six tasks justify a general conclusion, are correctness and generalizability risks, not circularity. Therefore the appropriate finding is no significant circularity, with score 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

This is an empirical systems paper. No new mathematical axioms, entities, or fitted parameters are visible from the abstract. The load-bearing premises are domain-level: SSL features fairly represent the continuous paradigm, quantized versions represent the discrete paradigm, and six tasks support the general conclusion. All experimental details that would normally appear as ledger entries (hyperparameters, budgets, dataset splits) are invisible because the full text as supplied was unreadable.

assumptions (2)
  • domain assumption SSL-based continuous features and their quantized counterparts fairly operationalize the two paradigms being compared.
    The abstract compares 'SSL-based discrete and continuous features'; the conclusion generalizes only as far as this operationalization is accepted.
  • domain assumption Six spoken language understanding tasks are representative enough to support a general claim about 'various tasks'.
    The abstract generalizes from six tasks to a broad conclusion; task selection details are not visible in the abstract.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Speech Discrete Tokens or Continuous Features? A Comparative Analysis for Spoken Language Understanding in SpeechLLMs." pith.science (2026). https://pith.science/paper/EBKRLX26

@misc{pith2026250817863,
  author       = {Pith},
  title        = {Pith review of: Speech Discrete Tokens or Continuous Features? A Comparative Analysis for Spoken Language Understanding in SpeechLLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EBKRLX26}},
  note         = {Machine review of arXiv:2508.17863}
}
read the original abstract

With the rise of Speech Large Language Models (SpeechLLMs), two dominant approaches have emerged for speech processing: discrete tokens and continuous features. Each approach has demonstrated strong capabilities in audio-related processing tasks. However, the performance gap between these two paradigms has not been thoroughly explored. To address this gap, we present a fair comparison of self-supervised learning (SSL)-based discrete and continuous features under the same experimental settings. We evaluate their performance across six spoken language understanding-related tasks using both small and large-scale LLMs (Qwen1.5-0.5B and Llama3.1-8B). We further conduct in-depth analyses, including efficient comparison, SSL layer analysis, LLM layer analysis, and robustness comparison. Our findings reveal that continuous features generally outperform discrete tokens in various tasks. Each speech processing method exhibits distinct characteristics and patterns in how it learns and processes speech information. We hope our results will provide valuable insights to advance spoken language understanding in SpeechLLMs.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 22 canonical work pages

  1. [1]

    Siddhant Arora, Kai-Wei Chang, Chung-Ming Chien, Yifan Peng, Haibin Wu, Yossi Adi, Emmanuel Dupoux, Hung-Yi Lee, Karen Livescu, and Shinji Watanabe. 2025. https://arxiv.org/abs/2504.08528 On the landscape of spoken language models: A comprehensive survey . Preprint, arXiv:2504.08528

    arXiv 2025

  2. [2]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609

    arXiv 2023

  3. [3]

    Emanuele Bastianelli, Andrea Vanzo, Pawel Swietojanski, and Verena Rieser. 2020. Slurp: A spoken language understanding resource package. arXiv preprint arXiv:2011.13205

    arXiv 2020

  4. [4]

    Zalán Borsos, Raphaël Marinier, Vincent, and 1 others. 2023. Audiolm: a language modeling approach to audio generation. IEEE/ACM transactions on audio, speech, and language processing

    work page 2023

  5. [5]

    Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Sungbok Kazemzadeh, and Shrikanth S Narayanan. 2008. Iemocap: Interactive emotional dyadic motion capture database. Language resources and evaluation, 42

    work page 2008

  6. [6]

    Davide Caffagni, Federico Cocchi, Luca Barsellotti, Nicholas Moratelli, Sara Sarto, Lorenzo Baraldi, Lorenzo Baraldi, Marcella Cornia, and Rita Cucchiara. 2024. https://arxiv.org/abs/2402.12451 The revolution of multimodal large language models: A survey . Preprint, arXiv:2402.12451

    arXiv 2024

  7. [7]

    Xuankai Chang, Brian Yan, Kwanghee Choi, Jee-Weon Jung, Yichen Lu, Soumi Maiti, Roshan Sharma, Jiatong Shi, Jinchuan Tian, Shinji Watanabe, and 1 others. 2024. Exploring speech recognition, translation, and understanding with discrete speech units: A comparative study. In ICASSP. IEEE

    work page 2024

  8. [8]

    Guoguo Chen, Shuzhou Chai, Guanbo Wang, Jiayu Du, Junbo Zhang, and 1 others. 2021. Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio. arXiv preprint arXiv:2106.06909

    arXiv 2021

Show all 57 references
  1. [9]

    Qian Chen, Yunfei Chu, Zhifu Gao, Zerui Li, Kai Hu, Xiaohuan Zhou, Jin Xu, Ziyang Ma, Wen Wang, Siqi Zheng, and 1 others. 2023. Lauragpt: Listen, attend, understand, and regenerate audio with gpt. arXiv preprint arXiv:2310.04673

    2023 arXiv

  2. [10]

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Jinyu Liu, and Kanda. 2022. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing

    2022

  3. [11]

    Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, Chang Zhou, and Jingren Zhou. 2024. Qwen2-audio technical report. arXiv preprint arXiv:2407.10759

    2024 arXiv

  4. [12]

    Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. 2023. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:2311.07919

    2023 arXiv

  5. [13]

    Wenqian Cui, Dianzhi Yu, Xiaoqi Jiao, Ziqiao Meng, Guangyan Zhang, Qichao Wang, Yiwen Guo, and Irwin King. 2025. https://arxiv.org/abs/2410.03751 Recent advances in speech language models: A survey . Preprint, arXiv:2410.03751

    2025 arXiv

  6. [14]

    Alexandre D \'e fossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. 2022. High fidelity neural audio compression. arXiv preprint arXiv:2210.13438

    2022 arXiv

  7. [15]

    Avihu Dekel and Raul Fernandez. 2024. Exploring the benefits of tokenization of discrete acoustic units. arXiv preprint arXiv:2406.05547

    2024 arXiv

  8. [16]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  9. [17]

    Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave, and Neil Zeghidour. 2024. Moshi: a speech-text foundation model for real-time dialogue

    2024

  10. [18]

    Yuan Gong, Alexander H Liu, Hongyin Luo, Leonid Karlinsky, and James Glass. 2023 a . Joint audio and speech understanding. In ASRU, pages 1--8. IEEE

    2023

  11. [19]

    Yuan Gong, Hongyin Luo, Alexander H Liu, Leonid Karlinsky, and James Glass. 2023 b . Listen, think, and understand. arXiv preprint arXiv:2305.10790

    2023 arXiv

  12. [20]

    Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Ruslan Lakhotia, and Abdelrahman Mohamed. 2021. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM transactions on audio, speech, and language processing

    2021

  13. [21]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and 1 others. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  14. [22]

    Shengpeng Ji, Yifu Chen, Minghui Fang, Jialong Zuo, Jingyu Lu, Hanting Wang, Ziyue Jiang, Long Zhou, Shujie Liu, Xize Cheng, and 1 others. 2024. Wavchat: A survey of spoken dialogue models. arXiv preprint arXiv:2411.13577

    2024 arXiv

  15. [23]

    Shengpeng Ji, Ziyue Jiang, Wen Wang, Yifu Chen, Minghui Fang, Jialong Zuo, Qian Yang, Xize Cheng, Zehan Wang, Ruiqi Li, Ziang Zhang, Xiaoda Yang, Rongjie Huang, Yidi Jiang, Qian Chen, Siqi Zheng, and Zhou Zhao. 2025. https://arxiv.org/abs/2408.16532 Wavtokenizer: an efficient ...

    2025 arXiv

  16. [24]

    Sander Land and Max Bartolo. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.649 Fishing for magikarp: Automatically detecting under-trained tokens in large language models . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 116...

    2024 doi

  17. [25]

    Yixing Li, Ruobing Xie, Xingwu Sun, Yu Cheng, and Zhanhui Kang. 2025. https://arxiv.org/abs/2410.17081 Continuous speech tokenizer in text to speech . Preprint, arXiv:2410.17081

    2025 arXiv

  18. [26]

    Ziyang Ma, Guanrou Yang, Yifan Yang, Zhifu Gao, Jiaming Wang, Du, and 1 others. 2024. An embarrassingly simple approach for llm with strong asr capacity. arXiv preprint arXiv:2402.08846

    2024 arXiv

  19. [27]

    Kentaro Mitsui, Koh Mitsuda, Toshiaki Wakatsuki, Yukiya Hono, and Kei Sawada. 2024. https://arxiv.org/abs/2406.12428 Pslm: Parallel generation of text and speech with llms for low-latency spoken dialogue systems . Preprint, arXiv:2406.12428

    2024 arXiv

  20. [28]

    Pooneh Mousavi, Jarod Duret, Salah Zaiem, Luca Della Libera, Artem Ploujnikov, Cem Subakan, and Mirco Ravanelli. 2024. How should we extract discrete audio tokens from self-supervised models? arXiv preprint arXiv:2406.10735

    2024 arXiv

  21. [29]

    Tu Anh Nguyen, Benjamin Muller, Bokai Yu, Marta R. Costa-jussa, Maha Elbayad, Sravya Popuri, Christophe Ropers, Paul-Ambroise Duquenne, Robin Algayres, Ruslan Mavlyutov, Itai Gat, Mary Williamson, Gabriel Synnaeve, Juan Pino, Benoit Sagot, and Emmanuel Dupoux. 2024. Spirit lm:...

    2024

  22. [30]

    OpenAI. 2023. Gpt-4 technical report

    2023

  23. [31]

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: an asr corpus based on public domain audio books. In ICASSP. IEEE

    2015

  24. [32]

    Jing Peng, Yucheng Wang, Yu Xi, Xu Li, Xizhuo Zhang, and Kai Yu. 2025. https://arxiv.org/abs/2410.18908 A survey on speech large language models . Preprint, arXiv:2410.18908

    2025

  25. [33]

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In ICML. PMLR

    2023

  26. [34]

    Paul K Rubenstein, Chulayuth Asawaroengchai, Nguyen, and 1 others. 2023. Audiopalm: A large language model that can speak and listen. arXiv preprint arXiv:2306.12925

    2023 arXiv

  27. [35]

    Roshan Sharma, Hira Dhamyal, Bhiksha Raj, and Rita Singh. 2022. Unifying the discrete and continuous emotion labels for speech emotion recognition

    2022

  28. [36]

    Suwon Shon, Kwangyoun Kim, Yi-Te Hsu, Prashant Sridhar, Shinji Watanabe, and Karen Livescu. 2024. Discreteslu: A large language model with self-supervised discrete speech units for spoken language understanding. arXiv preprint arXiv:2406.09345

    2024 arXiv

  29. [37]

    Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. 2023. Salmonn: Towards generic hearing abilities for large language models. arXiv preprint arXiv:2310.13289

    2023 arXiv

  30. [38]

    Benjamin van Niekerk, Marc-Andre Carbonneau, Julian Zaidi, Matthew Baas, Hugo Seute, and Herman Kamper. 2022. A comparison of discrete and soft speech units for improved voice conversion. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Proce...

    2022

  31. [39]

    Emmanuel Vincent and 1 others. 2016. The 4th chime speech separation and recognition challenge. http://spandh.dcs.shef.ac.uk/chime_challenge. Last accessed on 1 August, 2018

    2016

  32. [40]

    Chen Wang, Minpeng Liao, Zhongqiang Huang, Jinliang Lu, Junhong Wu, Yuchen Liu, Chengqing Zong, and Jiajun Zhang. 2023 a . Blsp: Bootstrapping language-speech pre-training via behavior alignment of continuation writing. arXiv preprint arXiv:2309.00916

    2023 arXiv

  33. [41]

    Dingdong Wang, Jin Xu, Ruihang Chu, Zhifang Guo, Xiong Wang, Jincenzi Wu, Dongchao Yang, Shengpeng Ji, and Junyang Lin. 2025. https://arxiv.org/abs/2503.02769 Inserter: Speech instruction following with unsupervised interleaved pre-training . Preprint, arXiv:2503.02769

    2025 arXiv

  34. [42]

    Tianrui Wang, Long Zhou, and 1 others. 2023 b . Viola: Unified codec language models for speech recognition, synthesis, and translation. arXiv preprint arXiv:2305.16107

    2023 arXiv

  35. [43]

    Xiaofei Wang, Manthan Thakker, and 1 others. 2024. Speechx: Neural codec language model as a versatile speech transformer. IEEE/ACM Transactions on Audio, Speech, and Language Processing

    2024

  36. [44]

    Pete Warden. 2018. Speech commands: A dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209

    2018 arXiv

  37. [45]

    Zhifei Xie and Changqiao Wu. 2024 a . Mini-omni: Language models can hear, talk while thinking in streaming

    2024

  38. [46]

    Zhifei Xie and Changqiao Wu. 2024 b . https://arxiv.org/abs/2410.11190 Mini-omni2: Towards open-source gpt-4o with vision, speech and duplex capabilities . ArXiv, abs/2410.11190

    2024 arXiv

  39. [47]

    Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. 2025. https://arxiv.org/abs/2503.20215 Qwen2.5-omni technical report . Preprint, arXiv:2503.20215

    2025 arXiv

  40. [48]

    Yaoxun Xu, Shi-Xiong Zhang, Jianwei Yu, Zhiyong Wu, and Dong Yu. 2024. https://arxiv.org/abs/2409.00800 Comparing discrete and continuous space llms for speech recognition . Preprint, arXiv:2409.00800

    2024 arXiv

  41. [49]

    Dongchao Yang, Songxiang Liu, Haohan Guo, Jiankun Zhao, Yuanyuan Wang, Helin Wang, Zeqian Ju, Xubo Liu, Xueyuan Chen, Xu Tan, Xixin Wu, and Helen Meng. 2025. https://arxiv.org/abs/2504.10344 Almtokenizer: A low-bitrate and semantic-rich audio codec tokenizer for audio language...

    2025 arXiv

  42. [50]

    Rong Ye, Chengqi Zhao, Tom Ko, Chutong Meng, Tao Wang, Mingxuan Wang, and Jun Cao. 2022. Gigast: A 10,000-hour pseudo speech translation corpus. arXiv preprint arXiv:2204.03939

    2022 arXiv

  43. [51]

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2024. https://doi.org/10.1093/nsr/nwae403 A survey on multimodal large language models . National Science Review, 11(12)

    2024 doi

  44. [52]

    Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. 2021. Soundstream: An end-to-end neural audio codec. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30

    2021

  45. [53]

    Aohan Zeng, Zhengxiao Du, Mingdao Liu, Kedong Wang, Shengmin Jiang, Lei Zhao, Yuxiao Dong, and Jie Tang. 2024. https://arxiv.org/abs/2412.02612 Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot . Preprint, arXiv:2412.02612

    2024 arXiv

  46. [54]

    Dong Zhang, Shimin Li, Xin Zhang, Jun Zhan, Pengyu Wang, Yaqian Zhou, and Xipeng Qiu. 2023 a . Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities. arXiv preprint arXiv:2305.11000

    2023 arXiv

  47. [55]

    Xin Zhang, Dong Zhang, Shimin Li, Yaqian Zhou, and Xipeng Qiu. 2023 b . Speechtokenizer: Unified speech tokenizer for speech large language models. arXiv preprint arXiv:2308.16692

    2023 arXiv

  48. [56]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  49. [57]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.