Pith. sign in

REVIEW 5 cited by

Enhancing Low-Resource Language and Instruction Following Capabilities of Audio Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2409.10999 v2 pith:7IUYO5LZ submitted 2024-09-17 cs.CL cs.AIcs.SDeess.AS

Potsawee Manakul , Guangzhi Sun , Warit Sirichotedumrong , Kasima Tharnpipitchai , Kunat Pipatanakul This is my paper · ORCID
classification cs.CLcs.AIcs.SDeess.AS
keywords audiolanguagemodelsenglishlow-resourcemultilingualdatainstruction-following
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Audio language models process audio inputs using textual prompts for tasks like speech recognition and audio captioning. Although built on multilingual pre-trained components, most are trained primarily on English, limiting their usability for other languages. This paper evaluates audio language models on Thai, a low-resource language, and finds that they lack emergent cross-lingual abilities despite their multilingual foundations. To address this, we explore data mixtures that optimize audio language models for both a target language and English while integrating audio comprehension and speech instruction-following into a unified model. Our experiments provide insights into improving instruction-following in low-resource languages by balancing language-specific and multilingual training data. The proposed model, Typhoon-Audio, significantly outperforms existing open-source models and achieves performance comparable to state-of-the-art Gemini-1.5-Pro in both English and Thai.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AudioJudge: Understanding What Works in Large Audio Model Based Speech Evaluation

    cs.CL 2025-07 conditional novelty 7.0 of 10

    With prompt engineering (audio concatenation plus in-context examples), large audio models rank speech synthesis systems in line with human preferences, reaching up to 0.91 Spearman correlation.

  2. LLaSO: A Foundational Framework for Reproducible Research in Large Language and Speech Model

    cs.CL 2025-08 unverdicted novelty 6.0 of 10

    LLaSO releases a 3.8B speech-language model, 25.5M training instances, and an evaluation benchmark, claiming a normalized score of 0.72.

  3. Weakly Supervised Data Refinement and Flexible Sequence Compression for Efficient Thai LLM-based ASR

    cs.SD 2025-05 conditional novelty 6.0 of 10

    EThai-ASR combines a self-refined Zipformer encoder with a Thai LLM and reports SOTA CER on Thai test sets plus a cosine-similarity frame pruning that gives 1.5-2.1x speedups in some modes.

  4. Speechless: Speech Instruction Training Without Speech for Low Resource Languages

    eess.AS 2025-05 conditional novelty 5.0 of 10

    Fine-tuning an LLM on text instructions converted to Whisper semantic tokens enables it to understand spoken instructions at inference, bypassing TTS and speech instruction data.

  5. Breaking the Barriers of Text-Hungry and Audio-Deficient AI

    cs.SD 2025-06 reject novelty 4.0 of 10

    A proposed audio-native translation framework called MAST with fractional diffusion is described, but no evidence is given that it produces working translations.

Pith tools