Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

Diarization-Aware Multi-Speaker Automatic Speech Recognition via Large Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An LLM conditioned on per-utterance speaker embeddings plus start and end times can transcribe overlapping speakers and emit utterance-level timestamps in one pass, and the authors report 31.6% word error on the AliMeeting evaluation set…

desk verdict Triplet-enrollment LLM design is genuinely new and MLC-SLM results are strong, but the AliMeeting SOTA claim rests on an unverified SD-CER/cpWER equivalence. read the letter →

arxiv 2506.05796 v1 pith:AG7HIORX submitted 2025-06-06 eess.AS

Yuke Lin , Ming Cheng , Ze Li , Beilong Tang , Ming Li This is my paper · ORCID
classification eess.AS
keywords multi-speakerautomaticspeechrecognitionlargelanguagemodelsspeakerdiarizationspeaker-attributedASRoverlappedtcpWERmeetingtranscriptionmultilingual
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a diarization-aware multi-speaker ASR system in which a large language model receives structured triplets for each target utterance: a speaker embedding, a normalized start time, and a normalized end time. The central claim is that these triplets let a single model jointly transcribe overlapping speakers and produce utterance-level timestamps, something serialized-output and target-speaker ASR systems do not do. On a Mandarin meeting benchmark the system reports 31.6% word error on the evaluation set and 32.17% time-constrained word error, and on a multilingual two-speaker challenge it reports time-constrained word error dropping from 76.12% to 24.95% on development and from 60.39% to 20.44% on test. A fair reader would care because this points toward one model that outputs who-spoke-what-and-when rather than separate modules whose errors propagate.

What carries the argument

The load-bearing object is the triplet enrollment instruction: for each target utterance, the model receives a speaker embedding, a normalized start time, and a normalized end time, linearly projected into the LLM's input space. These triplets tell the decoder whose words to write and during which interval, and because several triplets are fed at once, multiple speakers can be decoded jointly with cross-utterance context. The second mechanism is a gated cross-attention step between semantic speech features and speaker features, which modulates how much speaker identity affects each frame before an adapter projects the features into the LLM. Three data augmentations—replacing an embedding and blanking its label, dropping whole triplets, and shuffling triplet order—train the model to ignore faulty diarization information and to follow the enrollment order.

What would settle it

Re-score the system's hypotheses on the AliMeeting evaluation set with the character-level error metric and reference normalization used by the published baselines; if the score does not beat 31.8%, the state-of-the-art claim fails. As a separate check, replace the predicted diarization boundaries with oracle boundaries and see whether the time-constrained word error improves; if it does not, the timing signal is not doing the work attributed to it.

Watch

Extended reading notes

Core claim

The paper's central discovery is that speaker identity and utterance timing can be packaged as conditioning inputs for an LLM, and that the LLM will then transcribe the enrolled speaker's words in the indicated interval while also returning the interval boundaries. The architecture passes frame-level semantic features and frame-level speaker features through a gated cross-attention step, projects the merged features and the triplet embeddings into the LLM's input space, and fine-tunes the LLM with low-rank adapters. Trained on synthetic mixtures plus real meeting data, the system reports 31.6% constrained permutation word error (cpWER) on the AliMeeting evaluation set and 35.1% on the test set, with 32.17% and 36.36% time-constrained permutation word error (tcpWER); on MLC-SLM Task 2 it reports 24.95% tcpWER on development and 20.44% on test against official baselines of 76.12% and 60.39%. The authors also report that tcpWER rises with speaker count, from 14.94% with two speakers to 41.59% with four on the evaluation set.

Load-bearing premise

The paper's evaluation section equates its word-error metric with the character-error metric used by earlier systems, and the claimed state-of-the-art ranking depends on that equation being correct.

Editorial extensions

If this is right

  • The model emits utterances with timestamps aligned to the diarization output, so meetings can receive who-spoke-what-and-when in a single decoding pass.
  • Because it reads multiple triplets at once, the LLM can exploit cross-speaker context when decoding overlapping speech, reducing the need for explicit speech separation.
  • On the multilingual challenge the reported tcpWER fell by more than half in every language shown, suggesting the triplet interface transfers across languages without language-specific adaptation.
  • Error degrades gracefully with speaker count, from 14.94% to 41.59% tcpWER on the evaluation set, so the approach remains usable in four-speaker meetings.
  • The 32.17% tcpWER on the AliMeeting evaluation set, a metric the paper says it is the first to report there, gives later systems a concrete joint accuracy-plus-timing number to beat.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A test the paper does not run: feed several diarization hypotheses as triplets and let the LLM keep the transcription with the highest confidence, which could make the whole cascade robust to diarization misses without retraining.
  • A testable extension is to replace predicted diarization boundaries with oracle boundaries at evaluation; if tcpWER drops substantially, the remaining error is diarization-driven rather than transcription-driven, a distinction the current single-condition run cannot expose.
  • Because the gated cross-attention merges speaker and semantic features frame-by-frame, the same interface could plausibly be reused for target-speaker extraction or speaker-conditioned summarization by changing only the instruction text, though the paper does not explore those tasks.
  • An implicit consequence of reporting tcpWER first is that it sets a new comparison standard for speaker-attributed ASR: future systems that optimize content alone will not be comparable unless they also commit to timestamps.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper presents a semi-end-to-end multi-speaker ASR system built around a large language model. The audio is processed by a frozen Whisper encoder and the speaker information by a ResNet34 encoder; the LLM (Qwen2.5-3B, LoRA-tuned) is conditioned on 'triplets' consisting of a target-speaker embedding and normalized start/end times, and is trained to generate transcriptions for each enrolled speaker segment. The authors introduce gated cross-attention between speech and speaker features, three training-time augmentations (embedding replacement, embedding dropout, triplet shuffling), and chunk-based inference. The system is pretrained on synthetic English mixtures built from Common Voice and VoxBlink2 (the latter transcribed by Whisper) and fine-tuned on AliMeeting and the MLC-SLM challenge. Results include a claimed state-of-the-art cpWER of 31.6% on AliMeeting Eval, tcpWER of 32.17%/36.36% on AliMeeting Eval/Test, and large tcpWER reductions on MLC-SLM (24.95% vs 76.12% Dev, 20.44% vs 60.39% Test).

Significance. If the results are taken at face value, the MLC-SLM tcpWER results are a clear positive: the system reduces the official baseline by more than 50% absolute, and the per-language table shows consistent gains. The AliMeeting SOTA claim, however, is not yet supported because the comparison mixes cpWER and SD-CER. The architectural idea of conditioning an LLM on speaker-identity/time triplets is interesting and potentially useful for timestamped meeting transcription, but the paper does not isolate the contribution of this mechanism. The paper is generally well structured and reports useful implementation details (LoRA rank, augmentation probabilities, chunking limits), and the use of meeteval's tcpWER is a step toward evaluating timestamps jointly with content. No code is stated to be available, which would be a useful addition for reproducibility.

major comments (3)
  1. [III-D and Table I] The claimed state-of-the-art result on AliMeeting Eval rests on the assertion in Section III-D that 'we treat cpWER and SD-CER as equivalent in this context.' This equivalence is not established and is likely false for Mandarin: cpWER as implemented in meeteval operates on word/token alignments, whereas the SD-CER numbers cited for FD-SOT, WD-SOT, CASA-ASR, and SA-Paraformer are character-level under the official AliMeeting protocol. For Chinese, character and word error counts can differ substantially. Moreover, the Eval-set margin over CASA-ASR is only 0.2% (31.6 vs 31.8), which is well within the plausible difference between the two metrics. The Test-set result (35.1%) is already worse than CASA-ASR (34.7%), so the Eval-set comparison is the sole basis for the SOTA claim. The authors should report SD-CER (or cpCER, as appropriate) for their own system using the official scoring tool, or report cpWER for all baselines, and state the metric in the table header and per row. Without this, the central quantitative claim is unsupported.
  2. [IV and Table II] The AliMeeting tcpWER results are presented with no comparison system. The text says the system produces 'high-quality utterance-level timestamps' and 'strong tcpWER results', but Table II contains only the proposed system's numbers. Since no prior tcpWER numbers exist on this benchmark, absolute values alone do not support the quality claim; a system could achieve low tcpWER simply by copying diarization segment boundaries with poor transcription, or high tcpWER due to an overly strict segmentation convention. The authors should compare against a strong cascaded baseline (e.g., a diarizer plus a standard ASR with forced alignment) or at least report an oracle-timing upper bound and a no-timing baseline, and discuss the 5-second tolerance's interaction with their segment-level timestamps. Until then, the timestamping contribution is not demonstrated relative to alternatives.
  3. [III-C and IV] The paper attributes its gains to the triplet enrollment mechanism, the gated cross-attention, and the three data-augmentation strategies, but no ablation isolates any of these components. For example, the reader cannot tell whether conditioning on start/end times improves over speaker embeddings alone, whether the gating in Eq. (2) matters beyond a simple concatenation, or whether the augmentation probabilities (0.05/0.1/0.2) were tuned. Given that the backbone is a 3B-parameter LLM, the relative contribution of the proposed architecture could be small. A minimal set of ablations on MLC-SLM Dev (e.g., speaker-only enrollment vs. full triplet, with and without augmentation) would substantiate the design claims and is within the scope of the current experiments.
minor comments (6)
  1. [Table I] The acronym 'f&i-speaker' in the SA-Paraformer row is undefined; please spell out what 'f&i' refers to.
  2. [II-A, Eq. (2)] The gating matrix Wg is not defined; specify its dimensions and whether a bias term is included.
  3. [III-C2] The text mentions 'per-speaker utterance limits' and 'variable-length sampling windows' but gives no numerical values; please provide the actual constraints for reproducibility.
  4. [II-D] The chunking configurations (segment limit 10 with 4 segments per speaker for AliMeeting; 8 total segments with 6 per speaker for MLC-SLM) are stated without rationale; a sentence explaining how these were chosen would help readers assess sensitivity.
  5. [III-C2] There are minor typographical issues: 'V oxBlink2' should be 'VoxBlink2', 'pshuf f le' should be 'pshuffle', and 'Alimeeting' is inconsistently capitalized.
  6. [Table III] The Test column is empty for the per-language rows; please add a note stating that per-language Test scores are not released by the official evaluation server.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: results are direct empirical measurements, and the metric-equivalence caveat in Section III-D is a comparability risk, not a circular derivation.

full rationale

The paper's claims are empirical rather than derived: cpWER and tcpWER values are measured on AliMeeting and MLC-SLM against external ground truth and published or official baselines, and the LLM outputs are generated transcriptions, not quantities fitted from the target metrics. The only self-citation is the authors' S2SND diarization front-end [34], [35], whose outputs feed the triplet enrollment; S2SND is itself evaluated on the same benchmarks with DER, so it is independently checkable and does not make the ASR result circular. The one genuinely load-bearing caveat is Section III-D, which states 'we treat cpWER and SD-CER as equivalent in this context' while Table I compares the proposed system's cpWER with baselines that reported SD-CER; if that equivalence fails, the Eval-set state-of-the-art comparison is unsupported. That is a metric-comparability and correctness problem, however, not a circular reduction: no equation or parameter is defined in terms of the claimed outcome.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper relies on several domain assumptions: metric equivalence (SD-CER vs cpWER), the accuracy of the self-developed S2SND diarization front-end, the reliability of Whisper pseudo-labels for pretraining, and the transferability of synthetic mixtures to real meetings. These are not derived or independently validated in the paper. Free parameters are limited to hand-set hyperparameters for data augmentation, LoRA, and chunking; none are fitted to the test sets in a way that would invalidate the results, but the lack of ablations makes their contribution unverified.

free parameters (5)
  • Embedding replacement probability = 0.05
    Hand-chosen data augmentation hyperparameter that simulates incorrect speaker embeddings during training.
  • Embedding dropout probability = 0.1
    Hand-chosen data augmentation hyperparameter that simulates missing diarization outputs.
  • Triplet shuffling probability = 0.2
    Hand-chosen data augmentation hyperparameter that shuffles enrollment order to enforce order-invariant decoding.
  • LoRA rank and alpha = rank=8, alpha=16
    Standard LoRA hyperparameters selected by hand for parameter-efficient fine-tuning; not fitted to evaluation data.
  • Chunk segment limits = AliMeeting: 10 segments, 4 per speaker; MLC-SLM: 8 segments, 6 per speaker
    Inference-time configuration tuned per evaluation set to balance context and error propagation.
assumptions (4)
  • domain assumption SD-CER and cpWER are equivalent evaluation metrics for speaker-attributed ASR.
    Stated in Section III-D without justification; the basis for comparing against prior baselines reported with SD-CER.
  • domain assumption S2SND diarization outputs (speaker embeddings and time boundaries) are sufficiently accurate for triplet enrollment.
    The framework relies on diarization to produce triplets; no analysis of error propagation from diarization to ASR is provided.
  • domain assumption Whisper-Large-V3 transcriptions of VoxBlink2 are reliable ground truth for pretraining.
    Approximately 4 million utterance-text pairs are generated with Whisper-Large-V3; errors in these pseudo-labels propagate into the model.
  • domain assumption Synthetic mixtures of utterances from different speakers simulate real meeting conditions.
    Pretraining uses on-the-fly mixing of CommonVoice and VoxBlink2; transfer to real far-field meetings is assumed without direct validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Diarization-Aware Multi-Speaker Automatic Speech Recognition via Large Language Models." pith.science (2026). https://pith.science/paper/AG7HIORX

@misc{pith2026250605796,
  author       = {Pith},
  title        = {Pith review of: Diarization-Aware Multi-Speaker Automatic Speech Recognition via Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AG7HIORX}},
  note         = {Machine review of arXiv:2506.05796}
}
read the original abstract

Multi-speaker automatic speech recognition (MS-ASR) faces significant challenges in transcribing overlapped speech, a task critical for applications like meeting transcription and conversational analysis. While serialized output training (SOT)-style methods serve as common solutions, they often discard absolute timing information, limiting their utility in time-sensitive scenarios. Leveraging recent advances in large language models (LLMs) for conversational audio processing, we propose a novel diarization-aware multi-speaker ASR system that integrates speaker diarization with LLM-based transcription. Our framework processes structured diarization inputs alongside frame-level speaker and semantic embeddings, enabling the LLM to generate segment-level transcriptions. Experiments demonstrate that the system achieves robust performance in multilingual dyadic conversations and excels in complex, high-overlap multi-speaker meeting scenarios. This work highlights the potential of LLMs as unified back-ends for joint speaker-aware segmentation and transcription.

Figures

Figures reproduced from arXiv: 2506.05796 by the authors.

Figure 1
Figure 1. An overview of our framework. introducing a triplet-based enrollment mechanism, where each enrolled target speaker is represented by a tuple consisting of (i) a speaker embedding, (ii) a start time of sentence, and (iii) an end time of sentence. This representation allows the model to jointly transcribe speech and output utterance￾level timestamps that are directly aligned with a diarization system. Furthermore, we … view at source ↗
Figure 2
Figure 2. The construction of our inputs (three sentences from two speakers). III. EXPERIMENTAL SETTINGS A. Dataset Usage To support the pretraining of large-scale neural networks for multi-speaker ASR, we construct a synthetic dataset using two publicly available corpora: Common Voice [30] and VoxBlink2 [31], [32]. Common Voice is a crowd-sourced multilingual ASR dataset; we use only its English subset, which provides high-q… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TagSpeech: End-to-End Multi-Speaker ASR and Diarization with Fine-Grained Temporal Grounding

    eess.AS 2026-01 conditional novelty 6.0 of 10

    A frozen-LLM speech system with dual semantic/speaker encoders and interleaved numeric time anchors jointly predicts transcript, speaker, and timestamps, beating zero-shot Gemini/Qwen baselines on AMI and AliMeeting DER.

  2. The DKU System for Multi-Speaker Automatic Speech Recognition in MLC-SLM Challenge

    eess.AS 2025-07 conditional novelty 4.0 of 10

    A challenge system combining speaker diarization, speaker embeddings, and a Qwen2.5 LLM adapter architecture reports 18.08% tcpWER on multilingual multi-speaker ASR, far below the 60.39% baseline.

Reference graph

Works this paper leans on

40 extracted references · 25 canonical work pages · cited by 2 Pith papers

  1. [1]

    Conformer: Convolution- augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, and R. Pang, “Conformer: Convolution- augmented transformer for speech recognition,” in Proc. Interspeech , 2020, pp. 5036–5040

    2020

  2. [2]

    Recent advances in end-to-end automatic speech recog- nition,

    J. Li et al. , “Recent advances in end-to-end automatic speech recog- nition,” APSIPA Transactions on Signal and Information Processing , vol. 11, no. 1, 2022

    work page 2022

  3. [3]

    Zipformer: A faster and better encoder for automatic speech recognition,

    Z. Yao, L. Guo, X. Yang, W. Kang, F. Kuang, Y . Yang, Z. Jin, L. Lin, and D. Povey, “Zipformer: A faster and better encoder for automatic speech recognition,” in Proc. ICLR, 2024

    2024

  4. [4]

    Survey of end-to-end multi-speaker au- tomatic speech recognition for monaural audio,

    X. He and J. Whitehill, “Survey of end-to-end multi-speaker au- tomatic speech recognition for monaural audio,” arXiv preprint arXiv:2505.10975, 2025

    arXiv 2025

  5. [5]

    The chime-7 dasr challenge: Distant meeting transcription with multiple devices in diverse scenarios,

    S. Cornell, M. Wiesner, S. Watanabe, D. Raj, X. Chang, P. Garcia, M. Maciejewski, Y . Masuyama, Z.-Q. Wang, S. Squartini et al. , “The chime-7 dasr challenge: Distant meeting transcription with multiple devices in diverse scenarios,” arXiv preprint arXiv:2306.13734 , 2023

    arXiv 2023

  6. [6]

    The chime-7 challenge: System description and performance of nemo team’s dasr system,

    T. J. Park, H. Huang, A. Jukic, K. Dhawan, K. C. Puvvada, N. Koluguri, N. Karpov, A. Laptev, J. Balam, and B. Ginsburg, “The chime-7 challenge: System description and performance of nemo team’s dasr system,” arXiv preprint arXiv:2310.12378 , 2023

    arXiv

  7. [7]

    The iacas- thinkit system for chime-7 challenge,

    L. Ye, H. Lu, G. Cheng, Y . Chen, Z. Shang, and X. Li, “The iacas- thinkit system for chime-7 challenge,” in 7th International Workshop on Speech Processing in Everyday Environments (CHiME 2023) , 2023, pp. 23–26

    work page 2023

  8. [8]

    Lexical speaker error correction: Leveraging language models for speaker diarization error correction,

    R. Paturi, S. Srinivasan, and X. Li, “Lexical speaker error correction: Leveraging language models for speaker diarization error correction,” in Proc. Interspeech, 2023, pp. 3567–3571

    work page 2023

Show all 40 references
  1. [9]

    Diariza- tionlm: Speaker diarization post-processing with large language models,

    Q. Wang, Y . Huang, G. Zhao, E. Clark, W. Xia, and H. Liao, “Diariza- tionlm: Speaker diarization post-processing with large language models,” in Proc. Interspeech, 2024, pp. 3754–3758

    2024

  2. [10]

    Continuous speech separation with conformer,

    S. Chen, Y . Wu, Z. Chen, J. Wu, J. Li, T. Yoshioka, C. Wang, S. Liu, and M. Zhou, “Continuous speech separation with conformer,” in Proc. ICASSP. IEEE, 2021, pp. 5749–5753

    2021

  3. [11]

    Low latency online blind source separation based on joint optimization with blind dereverberation,

    T. Ueda, T. Nakatani, R. Ikeshita, K. Kinoshita, S. Araki, and S. Makino, “Low latency online blind source separation based on joint optimization with blind dereverberation,” in Proc. ICASSP. IEEE, 2021, pp. 506– 510

    2021

  4. [12]

    Gpu-accelerated guided source separation for meeting transcription,

    D. Raj, D. Povey, and S. Khudanpur, “Gpu-accelerated guided source separation for meeting transcription,” in Proc. Interspeech , 2023, pp. 3507–3511

    2023

  5. [13]

    Recognizing multi-talker speech with permutation invariant training,

    D. Yu, X. Chang, and Y . Qian, “Recognizing multi-talker speech with permutation invariant training,” in Proc. Interspeech, 2017, pp. 2456– 2460

    2017

  6. [14]

    Mimo- speech: End-to-end multi-channel multi-speaker speech recognition,

    X. Chang, W. Zhang, Y . Qian, J. Le Roux, and S. Watanabe, “Mimo- speech: End-to-end multi-channel multi-speaker speech recognition,” in Proc. ASRU. IEEE, 2019, pp. 237–244

    2019

  7. [15]

    Improving end-to- end single-channel multi-talker speech recognition,

    W. Zhang, X. Chang, Y . Qian, and S. Watanabe, “Improving end-to- end single-channel multi-talker speech recognition,” IEEE/ACM Trans- actions on Audio, Speech, and Language Processing , vol. 28, pp. 1385– 1394, 2020

    2020

  8. [16]

    Serialized output training for end-to-end overlapped speech recognition,

    N. Kanda, Y . Gaur, X. Wang, Z. Meng, and T. Yoshioka, “Serialized output training for end-to-end overlapped speech recognition,” in Proc. Interspeech, 2020, pp. 2797–2801

    2020

  9. [17]

    End-to-end speaker-attributed asr with transformer,

    N. Kanda, G. Ye, Y . Gaur, X. Wang, Z. Meng, Z. Chen, and T. Yoshioka, “End-to-end speaker-attributed asr with transformer,” in Proc. Inter- speech, 2021, pp. 4413–4417

    2021

  10. [19]

    Streaming speaker-attributed asr with token-level speaker embeddings,

    N. Kanda, J. Wu, Y . Wu, X. Xiao, Z. Meng, X. Wang, Y . Gaur, Z. Chen, J. Li, and T. Yoshioka, “Streaming speaker-attributed asr with token-level speaker embeddings,” in Proc. Interspeech, 2022, pp. 521–525

    2022

  11. [20]

    Auxiliary interference speaker loss for target-speaker speech recognition,

    N. Kanda, S. Horiguchi, R. Takashima, Y . Fujita, K. Nagamatsu, and S. Watanabe, “Auxiliary interference speaker loss for target-speaker speech recognition,” in Proc. Interspeech, 2019, pp. 236–240

    2019

  12. [21]

    Conformer- based target-speaker automatic speech recognition for single-channel audio,

    Y . Zhang, K. C. Puvvada, V . Lavrukhin, and B. Ginsburg, “Conformer- based target-speaker automatic speech recognition for single-channel audio,” in Proc. ICASSP. IEEE, 2023, pp. 1–5

    2023

  13. [22]

    Extending whisper with prompt tuning to target-speaker asr,

    H. Ma, Z. Peng, M. Shao, J. Li, and J. Liu, “Extending whisper with prompt tuning to target-speaker asr,” in Proc. ICASSP. IEEE, 2024, pp. 12 516–12 520

    2024

  14. [23]

    Adapting self- supervised models to multi-talker speech recognition using speaker embeddings,

    Z. Huang, D. Raj, P. Garc ´ıa, and S. Khudanpur, “Adapting self- supervised models to multi-talker speech recognition using speaker embeddings,” in Proc. ICASSP. IEEE, 2023, pp. 1–5

    2023

  15. [24]

    Empowering whisper as a joint multi-talker and target-talker speech recognition system,

    L. Meng, J. Kang, Y . Wang, Z. Jin, X. Wu, X. Liu, and H. Meng, “Empowering whisper as a joint multi-talker and target-talker speech recognition system,” in Proc. Interspeech, 2024, pp. 4653–4657

    2024

  16. [25]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  17. [26]

    Deepseek-v3 technical report,

    A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan et al., “Deepseek-v3 technical report,” arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  18. [27]

    Qwen3 technical report,

    A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv et al. , “Qwen3 technical report,” arXiv preprint arXiv:2505.09388, 2025

    2025 arXiv

  19. [28]

    M2met: The icassp 2022 multi-channel multi- party meeting transcription challenge,

    F. Yu, S. Zhang, Y . Fu, L. Xie, S. Zheng, Z. Du, W. Huang, P. Guo, Z. Yan, B. Ma et al. , “M2met: The icassp 2022 multi-channel multi- party meeting transcription challenge,” in Proc. ICASSP. IEEE, 2022, pp. 6167–6171

    2022

  20. [29]

    Meeteval: A toolkit for computation of word error rates for meeting transcription systems,

    T. v. Neumann, C. B. Boeddeker, M. Delcroix, and R. Haeb-Umbach, “Meeteval: A toolkit for computation of word error rates for meeting transcription systems,” in Proc. CHiME, 2023, pp. 27–32

    2023

  21. [30]

    Com- mon voice: A massively-multilingual speech corpus,

    R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, “Com- mon voice: A massively-multilingual speech corpus,” arXiv preprint arXiv:1912.06670, 2019

    1912 arXiv

  22. [31]

    V oxblink: A large scale speaker verification dataset on camera,

    Y . Lin, X. Qin, G. Zhao, M. Cheng, N. Jiang, H. Wu, and M. Li, “V oxblink: A large scale speaker verification dataset on camera,” in Proc. ICASSP, 2024, pp. 10 271–10 275

    2024

  23. [32]

    V oxblink2: A 100k+ speaker recognition corpus and the open-set speaker-identification benchmark,

    Y . Lin, M. Cheng, F. Zhang, Y . Gao, S. Zhang, and M. Li, “V oxblink2: A 100k+ speaker recognition corpus and the open-set speaker-identification benchmark,” in Proc. Interspeech, 2024, pp. 4263–4267

    2024

  24. [33]

    Robust speech recognition via large-scale weak super- vision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak super- vision,” in Proc. ICML. PMLR, 2023, pp. 28 492–28 518

    2023

  25. [34]

    Sequence-to-sequence neural diarization with automatic speaker detection and representation,

    M. Cheng, Y . Lin, and M. Li, “Sequence-to-sequence neural diarization with automatic speaker detection and representation,” arXiv preprint arXiv:2411.13849, 2024

    2024 arXiv

  26. [35]

    Target-speaker voice activity detection via sequence-to-sequence prediction,

    M. Cheng, W. Wang, Y . Zhang, X. Qin, and M. Li, “Target-speaker voice activity detection via sequence-to-sequence prediction,” in Proc. ICASSP. IEEE, 2023, pp. 1–5

    2023

  27. [36]

    Qwen2.5: A party of foundation models,

    Q. Team, “Qwen2.5: A party of foundation models,” September 2024. [Online]. Available: https://qwenlm.github.io/blog/qwen2.5/

    2024

  28. [37]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. CVPR, 2016, pp. 770–778

    2016

  29. [38]

    Osum: Advancing open speech un- derstanding models with limited resources in academia,

    X. Geng, K. Wei, Q. Shao, S. Liu, Z. Lin, Z. Zhao, G. Li, W. Tian, P. Chen, Y . Li et al. , “Osum: Advancing open speech un- derstanding models with limited resources in academia,” arXiv preprint arXiv:2501.13306, 2025

    2025 arXiv

  30. [39]

    A comparative study on speaker-attributed automatic speech recognition in multi-party meet- ings,

    F. Yu, Z. Du, S. Zhang, Y . Lin, and L. Xie, “A comparative study on speaker-attributed automatic speech recognition in multi-party meet- ings,” in Proc. Interspeech, 2022, pp. 560–564

    2022

  31. [40]

    Casa-asr: Context-aware speaker-attributed asr,

    M. Shi, Z. Du, Q. Chen, F. Yu, Y . Li, S. Zhang, J. Zhang, and L.-R. Dai, “Casa-asr: Context-aware speaker-attributed asr,” in Proc. Interspeech, 2023, pp. 411–415

    2023

  32. [41]

    Sa-paraformer: Non-autoregressive end-to-end speaker-attributed asr,

    Y . Li, F. Yu, Y . Liang, P. Guo, M. Shi, Z. Du, S. Zhang, and L. Xie, “Sa-paraformer: Non-autoregressive end-to-end speaker-attributed asr,” in Proc. ASRU, 2023, pp. 1–7

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.