Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Self-Improvement for Audio Large Language Model using Unlabeled Speech

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Attention saliency can serve as a label-free reward for improving audio LLMs, cutting word error rate by up to 20.8%.

desk verdict Novel saliency-based unsupervised reward for audio LLM self-improvement, but the label-free saliency computation is under-specified and the empirical claims are slightly overstated; the core idea deserves a careful major revision. read the letter →

arxiv 2507.20169 v1 pith:QXDAPKGI submitted 2025-07-27 cs.SD eess.AS

Shaowen Wang , Xinyuan Chen , Yao Xu This is my paper · ORCID
classification cs.SDeess.AS
keywords audiolargelanguagemodelsself-improvementunsuperviseddomainadaptationattentionsaliencyreinforcementlearningautomaticspeechrecognitionspeech-to-texttranslationspokenquestionanswering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SI-SDA, the method this paper proposes, asks whether a pre-trained audio large language model can improve itself on speech recognition, translation, and spoken question answering using only hours of unlabeled audio from the target domain. The paper claims the answer is yes: while decoding, the model's attention to prompt tokens versus acoustic frames reveals how likely a generated hypothesis is to be wrong, and this signal can be used as a reward in reinforcement learning. The authors report consistent improvements over several unsupervised baselines, including a 20.8% relative word error rate (WER) reduction on TED-3 and BLEU gains on CoVoST2 and LibriSQA, using only 2–5 hours of unlabeled speech. The broader consequence is that audio LLMs may not need curated labels or source-domain data to adapt to noisy or accented conditions, which would make real-world deployment cheaper.

What carries the argument

The central object is the saliency-based quality score $Q(\hat{Y})$: for each decoded token, the ratio of attention saliency spent on prompt tokens to total attention saliency, averaged over the whole hypothesis. Saliency here is a gradient-based importance measure over attention weights, computed via Taylor expansion in Eq. (2). The score serves as a label-free reward in a reinforcement-learning loop: each hypothesis in the beam receives an advantage $-(Q-\bar{Q})$, so training shifts probability mass toward hypotheses that align with acoustic content rather than falling back on the prompt. This is the mechanism that lets the model adapt itself without ground-truth transcripts or source-domain data.

What would settle it

On any target set with ground truth, rank the beam hypotheses by $Q(\hat{Y})$ and measure the correlation with WER or BLEU; if low-Q hypotheses are not systematically better, the reward in Eq. (10) should push probability toward worse hypotheses. A cleaner falsifying experiment would run SI-SDA on a domain whose errors come from lexical confusions rather than attention fallback; if WER does not improve while Q is still computed, the central assumption fails.

Watch

Extended reading notes

Core claim

The central claim is that the internal saliency of an audio LLM's decoding carries a domain-general quality signal: tokens decoded incorrectly tend to rely more heavily on prompt tokens than on acoustic frames. SI-SDA turns this into an unsupervised reward by averaging, over all positions of a candidate hypothesis, the fraction of attention saliency that lands on prompt tokens, yielding the score $Q(\hat{Y})$. The model generates an N-best beam set, assigns each hypothesis an advantage based on whether its Q lies below the beam average, and updates low-rank adapter parameters (LoRA) through the policy-gradient objective in Eq. (10) to raise the probability of low-Q hypotheses. With no labeled data and no need to recall source-domain data, the method reports consistent gains over zero-shot, self-training, filtering, confidence reweighting, and STAR baselines across CHiME-4, LibriSpeech with FreeSound noise, CommonVoice, Switchboard, and TED-3, plus BLEU gains on CoVoST2 and LibriSQA.

Load-bearing premise

The method rests on a single bet: that an audio LLM's attention pattern marks its own mistakes, so a candidate hypothesis that leans more on the prompt than on the audio is likely wrong. The reward also depends on a saliency score computed from a loss gradient, yet in the unlabeled setting the paper does not specify what target that loss uses; if either link fails in a new domain, the reinforcement signal trains the model toward worse outputs.

Editorial extensions

If this is right

  • If SI-SDA works as described, audio LLMs can be adapted to a new acoustic domain with only a few hours of unlabeled audio, removing the annotation bottleneck that typically blocks deployment in noisy or accented environments.
  • The reward should transfer to tasks where labels are not unique, because the paper reports BLEU gains in both speech-to-text translation and spoken question answering.
  • Because the model relies only on its own decoding and unlabeled speech, it can be updated as new target-domain audio arrives without keeping source-domain data.
  • The reported performance converges in roughly 2–5 hours, so the method can act as a fast first adaptation step before deciding whether labeled data collection is worthwhile.
  • On several test sets the adapted model approaches or ties the supervised fine-tuning upper bound, while on others it remains short of it, so the method narrows but does not eliminate the gap to labeled adaptation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension of the same logic is to test whether prompt-reliance is a generic failure signature across modalities: the attention-saliency ratio should also rank the quality of generated text or image captions in vision-language models.
  • One testable prediction is that the method's gains should scale with the frequency of prompt-fallback errors; domains where errors come from systematic acoustic confusions that do not change attention patterns should show little or no improvement.
  • The data-efficiency result suggests an iterative recipe: run SI-SDA on a new unlabeled corpus, use the improved model to produce better pseudo-labels, and repeat, turning the method into a fully autonomous continual-learning loop.
  • If Q reliably identifies low-quality hypotheses, it could be combined with diversity-based filtering to select which utterances to train on, potentially improving data efficiency even further.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SI-SDA, an unsupervised domain adaptation method for audio large language models. The method computes saliency scores from attention matrices and loss gradients, defines a sequence-level prompt-reliance score Q(Ŷ) that is intended to indicate hypothesis quality without labels, and then uses Q as a reward in a policy-gradient-style objective over a beam set, reinforcing low-Q hypotheses. Experiments on Qwen2-Audio cover ASR (CHiME-4, LS-Freesound, CommonVoice, Switchboard, TED-3), speech-to-text translation (CoVoST2), and spoken question answering (LibriSQA), reporting WER and BLEU improvements over zero-shot and several unsupervised baselines, with up to 20.8% relative WER reduction on TED-3.

Significance. If the central claim holds, the method is practically valuable: it is source-free, uses only unlabeled target-domain speech, requires no recall of source data, and reports consistent improvements across several tasks and datasets. The paper has notable strengths: it validates the method against external metrics (WER, BLEU) rather than only against its own reward, introduces no fitted constants, and demonstrates data efficiency. However, the main claimed mechanism depends on an unsupervised saliency signal whose computation is not fully specified, and the empirical support lacks statistical rigor. The contribution is interesting and within scope for a speech/audio venue, but the central derivation needs clarification and the significance claims need stronger evidence before publication.

major comments (4)
  1. [III-A, Eq. (2)] The saliency score in Eq. (2) is defined via ∂L(x)/∂A_{h,l}, but the paper never specifies what L(x) is when the input is unlabeled. Since the method explicitly avoids ground-truth transcripts, the loss used to compute the saliency map is undefined, so Eqs. (7)-(8) cannot be implemented as written. If the intended choice is cross-entropy with the model's own hypothesis as the target, then the saliency measures self-agreement rather than alignment with any external text, and the R_E > R_C evidence in Fig. 1 does not transfer directly because that analysis is computed against ground-truth tokens. Please state the exact loss used in the unsupervised setting and provide a direct validation that the resulting Q ordering correlates with WER/BLEU on held-out unlabeled data.
  2. [Tables I-III and abstract] The claim that SI-SDA 'consistently and significantly improves' performance is not supported by the reported numbers, which lack error bars, confidence intervals, significance tests, and multiple seeds. Several differences are very small, e.g., CommonVoice Africa WER 4.9 vs 5.0, LS-Freesound car 2.5 vs 2.4 (STAR), and Table III it-en BLEU 36.1 vs 36.1. Please report means over at least three runs or a paired significance test across utterances, and temper the abstract's 'significantly' unless such evidence is provided.
  3. [III-D, Eq. (10)] The optimization loop is under-specified: since Q and the saliency maps depend on the model parameters θ, the paper should state whether the beam search, Q computation, and policy-gradient update are performed once per epoch, per mini-batch, or with re-sampling at each step, and whether gradients flow through the saliency computation. This is needed for reproducibility and for assessing whether the observed gains come from the reward signal or from a specific implementation detail.
  4. [V-A and Fig. 1] The motivating evidence that erroneous tokens rely more heavily on prompt tokens (R_E > R_C) is presented for only two ASR test sets and with no numerical values or error bars; the caption refers to 'clean and noise ASR datasets' while the text mentions 'LibriSpeech test set LS-Freesound.' Since this analysis is the sole justification for using Q as a reward in S2TT and SQA, the paper should provide a broader token-level analysis or, at minimum, an utterance-level correlation between Q and external quality scores for each task family.
minor comments (5)
  1. [III-A] The notation X = {x1, x2, ..., x3} appears to be a typo; it should be x_L or similar.
  2. [IV-A] The model is called Qwen-Audio2 in the text but the reference [25] is the Qwen2-Audio technical report; please align the naming.
  3. [V-C] The dataset name 'LirbriSQA' is a typo; it should be LibriSQA.
  4. [Table III] The caption contains the typo 'RESUTLS'; also, for the it-en row, SI-SDA ties with STAR, which is worth noting in the text.
  5. [V-A] The description of Figure 1 is confusing: the text says 'both clean LibriSpeech test set LS-Freesound' but the figure and caption suggest a comparison across multiple datasets; please clarify what is plotted and on which data the normalization is performed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the proposed reward is validated against external WER/BLEU and no parameter is fitted to the target metric.

full rationale

The paper's derivation chain is: (i) define a saliency score from loss gradients in Eq. (2); (ii) observe, using ground-truth token labels, that erroneous tokens have higher prompt reliance RE than correct tokens RC; (iii) define an unsupervised hypothesis-level score Q(Yhat) in Eqs. (7)-(8) as average prompt reliance; (iv) use Q to compute advantages in Eq. (9) and an RL objective in Eq. (10); and (v) evaluate against external WER and BLEU. None of these steps makes the prediction equivalent to an input by construction. Q is not defined in terms of WER or BLEU, no parameter is fitted to the evaluation metric, and the sign convention in Eq. (9) is explicit and coherent. The motivating RE>RC observation is an empirical premise, not a tautology. The main concern is that Eq. (2) leaves the loss L(x) unspecified for unlabeled hypotheses, making the implementation of Q ambiguous and creating a correctness risk about whether saliency computed without ground truth retains its quality-ranking property. That is a substantive empirical gap, but it is not circularity in the sense of a prediction reducing to a fitted input or to a self-citation chain. The paper does not rely on load-bearing self-citations: the cited prior works are external baselines or standard references, and no uniqueness theorem from the authors is invoked. Accordingly, the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical or architectural entities. The central assumptions are the validity of the saliency proxy in the unsupervised setting, the implicit self-supervised gradient, and the generalization of the token-level correlation to all tested tasks. No constants are fitted to the target metrics; hyperparameters are reported but not part of the derivation.

assumptions (5)
  • standard math Saliency from Taylor expansion of the loss with respect to attention weights approximates token importance (Eq. 2).
    Adopted from Michel et al. [26] and Simonyan et al. [27] as standard practice for interpreting attention; this is a background tool, not a new postulate.
  • domain assumption In the unsupervised setting, the loss gradient in Eq. (2) is computed with the model's own generated tokens as targets.
    The paper does not state how L(x) is defined without ground truth. The only consistent reading is that the model's own decoding is used as the supervisory target, which is a load-bearing assumption for Eqs. (7)-(8).
  • domain assumption Prompt-reliance R(i) negatively correlates with token correctness, and sequence-level average Q(Y-hat) negatively correlates with hypothesis quality.
    Supported only by the R_E versus R_C comparison in Fig. 1 on two ASR test sets. The method assumes this holds for all target domains and for S2TT and SQA without retesting the correlation.
  • domain assumption The policy gradient objective in Eq. (10) with an average-Q baseline improves the policy toward low-Q hypotheses.
    Standard RL reasoning, but it depends on Q being the correct reward signal. If Q is not a faithful quality proxy, the optimization will reinforce the wrong hypotheses.
  • domain assumption LoRA tuning with rank 16 preserves the model's learned behavior while allowing domain adaptation.
    LoRA is a standard efficient fine-tuning choice, but its sufficiency for this adaptation task is not separately validated; it is a practical choice described in Section IV-B.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-Improvement for Audio Large Language Model using Unlabeled Speech." pith.science (2026). https://pith.science/paper/QXDAPKGI

@misc{pith2026250720169,
  author       = {Pith},
  title        = {Pith review of: Self-Improvement for Audio Large Language Model using Unlabeled Speech},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QXDAPKGI}},
  note         = {Machine review of arXiv:2507.20169}
}
read the original abstract

Recent audio LLMs have emerged rapidly, demonstrating strong generalization across various speech tasks. However, given the inherent complexity of speech signals, these models inevitably suffer from performance degradation in specific target domains. To address this, we focus on enhancing audio LLMs in target domains without any labeled data. We propose a self-improvement method called SI-SDA, leveraging the information embedded in large-model decoding to evaluate the quality of generated pseudo labels and then perform domain adaptation based on reinforcement learning optimization. Experimental results show that our method consistently and significantly improves audio LLM performance, outperforming existing baselines in WER and BLEU across multiple public datasets of automatic speech recognition (ASR), spoken question-answering (SQA), and speech-to-text translation (S2TT). Furthermore, our approach exhibits high data efficiency, underscoring its potential for real-world deployment.

Figures

Figures reproduced from arXiv: 2507.20169 by the authors.

Figure 1
Figure 1. The RE and RC comparison across clean and noise ASR datasets. All values are normalized such that their sum equals 1, allowing for a clearer observation of the relative proportions. zero-shot performance varies significantly, ranging from 3.2 to 25.6, our method consistently achieves notable improve￾ments, demonstrating the stability of SI-SDA across different scenarios. (3) Compared to SFT, SI-SDA converges faster,… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Multimodal Deep Learning Framework for Early Diagnosis of Liver Cancer via Optimized BiLSTM-AM-VMD Architecture

    cs.LG 2025-09 reject novelty 3.0 of 10

    The paper claims a BiLSTM-AM-VMD model achieves AUC 0.963 for early HCC diagnosis, but the evidence is undermined by contradictory dataset descriptions and missing artifacts.

Reference graph

Works this paper leans on

35 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [1]

    Language mod- els are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language mod- els are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  2. [2]

    Conformer: Convolution-augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu et al., “Conformer: Convolution-augmented transformer for speech recognition,” arXiv preprint arXiv:2005.08100 , 2020

    arXiv 2005

  3. [3]

    Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models,

    Y . Chu, J. Xu, X. Zhou, Q. Yang, S. Zhang, Z. Yan, C. Zhou, and J. Zhou, “Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models,” arXiv preprint arXiv:2311.07919 , 2023

    arXiv 2023

  4. [4]

    Wavllm: Towards robust and adaptive speech large language model,

    S. Hu, L. Zhou, S. Liu, S. Chen, L. Meng, H. Hao, J. Pan, X. Liu, J. Li, S. Sivasankaran et al. , “Wavllm: Towards robust and adaptive speech large language model,” arXiv preprint arXiv:2404.00656 , 2024

    arXiv 2024

  5. [5]

    Salmonn: Towards generic hearing abilities for large language models,

    C. Tang, W. Yu, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. Ma, and C. Zhang, “Salmonn: Towards generic hearing abilities for large language models,” arXiv preprint arXiv:2310.13289 , 2023

    arXiv 2023

  6. [6]

    Speechgpt- gen: Scaling chain-of-information speech generation,

    D. Zhang, X. Zhang, J. Zhan, S. Li, Y . Zhou, and X. Qiu, “Speechgpt- gen: Scaling chain-of-information speech generation,” arXiv preprint arXiv:2401.13527, 2024. This paper has been accepted to Interspeech 2025. This is a preprint version. TABLE I WER M AIN RESULT ON ASR TASK. T HE BASELINES ARE INTRODUCED IN SECTION IV-B. T HE PERCENTAGE VALUES IN THE B...

    arXiv 2024

  7. [7]

    Prompting large language models with speech recognition abilities,

    Y . Fathullah, C. Wu, E. Lakomkin, J. Jia, Y . Shangguan, K. Li, J. Guo, W. Xiong, J. Mahadeokar, O. Kalinli et al., “Prompting large language models with speech recognition abilities,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 13 351–13 355

    2024

  8. [8]

    Noise-robust speech recognition with 10 minutes unparalleled in-domain data,

    C. Chen, N. Hou, Y . Hu, S. Shirol, and E. S. Chng, “Noise-robust speech recognition with 10 minutes unparalleled in-domain data,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2022, pp. 4298–4302

    work page 2022

Show all 35 references
  1. [9]

    Large language models can self-improve,

    J. Huang, S. S. Gu, L. Hou, Y . Wu, X. Wang, H. Yu, and J. Han, “Large language models can self-improve,” arXiv preprint arXiv:2210.11610 , 2022

    2022 arXiv

  2. [10]

    Feature alignment by uncertainty and self-training for source-free unsupervised domain adaptation,

    J. Lee and G. Lee, “Feature alignment by uncertainty and self-training for source-free unsupervised domain adaptation,” Neural Networks, vol. 161, pp. 682–692, 2023

    2023

  3. [11]

    Self- critical sequence training for image captioning,

    S. J. Rennie, E. Marcheret, Y . Mroueh, J. Ross, and V . Goel, “Self- critical sequence training for image captioning,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 7008–7024

    2017

  4. [12]

    Mutual enhancement of large language and reinforcement learning models through bi-directional feedback mechanisms: A case study,

    S. Gu, “Mutual enhancement of large language and reinforcement learning models through bi-directional feedback mechanisms: A case study,” arXiv preprint arXiv:2401.06603 , 2024

    2024 arXiv

  5. [13]

    Opera: Alleviating hallucination in multi- modal large language models via over-trust penalty and retrospection- allocation,

    Q. Huang, X. Dong, P. Zhang, B. Wang, C. He, J. Wang, D. Lin, W. Zhang, and N. Yu, “Opera: Alleviating hallucination in multi- modal large language models via over-trust penalty and retrospection- allocation,” in Proc. CVPR, 2024

    2024

  6. [14]

    Unsupervised domain adaptation by back- propagation,

    Y . Ganin and V . Lempitsky, “Unsupervised domain adaptation by back- propagation,” in International conference on machine learning. PMLR, 2015, pp. 1180–1189

    2015

  7. [15]

    An unsupervised deep domain adaptation approach for robust speech recognition,

    S. Sun, B. Zhang, L. Xie, and Y . Zhang, “An unsupervised deep domain adaptation approach for robust speech recognition,” NeuroComputing, vol. 257, pp. 79–87, 2017

    2017

  8. [16]

    Self-taught recognizer: Toward unsupervised adaptation for speech foundation models,

    Y . Hu, C. Chen, C.-H. H. Yang, C. Qin, P.-Y . Chen, E. S. Chng, and C. Zhang, “Self-taught recognizer: Toward unsupervised adaptation for speech foundation models,” arXiv preprint arXiv:2405.14161 , 2024

    2024 arXiv

  9. [17]

    Speech translation with large language models: An industrial practice,

    Z. Huang, R. Ye, T. Ko, Q. Dong, S. Cheng, M. Wang, and H. Li, “Speech translation with large language models: An industrial practice,” arXiv preprint arXiv:2312.13585 , 2023

    2023 arXiv

  10. [18]

    Zero- shot spoken language understanding via large language models: A preliminary study,

    Z. Zhu, X. Cheng, H. An, Z. Wang, D. Chen, and Z. Huang, “Zero- shot spoken language understanding via large language models: A preliminary study,” in Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-C...

    2024

  11. [19]

    Speechverse: A large-scale generalizable audio language model,

    N. Das, S. Dingliwal, S. Ronanki, R. Paturi, Z. Huang, P. Mathur, J. Yuan, D. Bekal, X. Niu, S. M. Jayanthi et al. , “Speechverse: A large-scale generalizable audio language model,” arXiv preprint arXiv:2405.08295, 2024

    2024 arXiv

  12. [20]

    Audiopalm: A large language model that can speak and listen,

    P. K. Rubenstein, C. Asawaroengchai, D. D. Nguyen, A. Bapna, Z. Bor- sos, F. d. C. Quitry, P. Chen, D. E. Badawy, W. Han, E. Kharitonovet al., “Audiopalm: A large language model that can speak and listen,” arXiv preprint arXiv:2306.12925, 2023

    2023 arXiv

  13. [21]

    Lauragpt: Listen, attend, understand, and regenerate audio with gpt,

    Z. Du, J. Wang, Q. Chen, Y . Chu, Z. Gao, Z. Li, K. Hu, X. Zhou, J. Xu, Z. Ma et al., “Lauragpt: Listen, attend, understand, and regenerate audio with gpt,” arXiv preprint arXiv:2310.04673 , 2023

    2023 arXiv

  14. [22]

    Speechgpt: Empowering large language models with intrinsic cross- modal conversational abilities,

    D. Zhang, S. Li, X. Zhang, J. Zhan, P. Wang, Y . Zhou, and X. Qiu, “Speechgpt: Empowering large language models with intrinsic cross- modal conversational abilities,” arXiv preprint arXiv:2305.11000, 2023

    2023 arXiv

  15. [23]

    Spirit-lm: Interleaved spoken and written language model,

    T. A. Nguyen, B. Muller, B. Yu, M. R. Costa-Jussa, M. Elbayad, S. Popuri, C. Ropers, P.-A. Duquenne, R. Algayres, R. Mavlyutov et al. , “Spirit-lm: Interleaved spoken and written language model,” Transactions of the Association for Computational Linguistics , vol. 13, pp. 30–52, 2025

    2025

  16. [24]

    mslam: Massively multilingual joint pre-training for speech and text,

    A. Bapna, C. Cherry, Y . Zhang, Y . Jia, M. Johnson, Y . Cheng, S. Khanuja, J. Riesa, and A. Conneau, “mslam: Massively multilingual joint pre-training for speech and text,” arXiv preprint arXiv:2202.01374, 2022

    2022 arXiv

  17. [25]

    Qwen2-audio technical report,

    Y . Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y . Leng, Y . Lv, J. He, J. Lin et al. , “Qwen2-audio technical report,” arXiv preprint arXiv:2407.10759, 2024. This paper has been accepted to Interspeech 2025. This is a preprint version

    2024 arXiv

  18. [26]

    Are sixteen heads really better than one?

    P. Michel, O. Levy, and G. Neubig, “Are sixteen heads really better than one?” Advances in neural information processing systems , vol. 32, 2019

    2019

  19. [27]

    Deep inside convolutional networks: Visualising image classification models and saliency maps,

    K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolutional networks: Visualising image classification models and saliency maps,” arXiv preprint arXiv:1312.6034 , 2013

    2013 arXiv

  20. [28]

    Minimum word error rate training for attention-based sequence-to-sequence models,

    R. Prabhavalkar, T. N. Sainath, Y . Wu, P. Nguyen, Z. Chen, C.-C. Chiu, and A. Kannan, “Minimum word error rate training for attention-based sequence-to-sequence models,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2018, pp...

    2018

  21. [29]

    The 4th chime speech separation and recognition challenge,

    E. Vincent, S. Watanabe, J. Barker, and R. Marxer, “The 4th chime speech separation and recognition challenge,” URL: http://spandh. dcs. shef. ac. uk/chime challenge/(last accessed on 1 August, 2018) , 2016

    2018

  22. [30]

    Freesound technical demo,

    F. Font, G. Roma, and X. Serra, “Freesound technical demo,” in Proc. ACM MM, 2013, pp. 411–412

    2013

  23. [31]

    Com- mon voice: A massively-multilingual speech corpus,

    R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, “Com- mon voice: A massively-multilingual speech corpus,” arXiv preprint arXiv:1912.06670, 2019

    1912 arXiv

  24. [32]

    Single headed attention based sequence-to-sequence model for state-of-the-art results on switch- board,

    Z. T ¨uske, G. Saon, K. Audhkhasi, and et al, “Single headed attention based sequence-to-sequence model for state-of-the-art results on switch- board,” in arXiv preprint arXiv:2001.07263 , 2020

    2001 arXiv

  25. [33]

    Covost: A diverse multilingual speech-to-text translation corpus,

    C. Wang, J. Pino, A. Wu, and J. Gu, “Covost: A diverse multilingual speech-to-text translation corpus,” arXiv preprint arXiv:2002.01320 , 2020

    2002 arXiv

  26. [34]

    Librisqa: A novel dataset and framework for spoken question answering with large language models,

    Z. Zhao, Y . Jiang, H. Liu, Y . Wang, and Y . Wang, “Librisqa: A novel dataset and framework for spoken question answering with large language models,” IEEE Transactions on Artificial Intelligence , 2024

    2024

  27. [35]

    Direct preference optimization: Your language model is secretly a reward model,

    R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, “Direct preference optimization: Your language model is secretly a reward model,” Advances in Neural Information Processing Systems, vol. 36, 2024. This paper has been accepted to Interspeech 2025. This...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.