Pith. sign in

REVIEW 9 cited by

Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answering

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2503.11197 v4 pith:QCSSJS5F submitted 2025-03-14 cs.SD cs.AIcs.CLeess.AS

Gang Li , Jizhong Liu , Heinrich Dinkel , Yadong Niu , Junbo Zhang , Jian Luan This is my paper · ORCID
classification cs.SDcs.AIcs.CLeess.AS
keywords audioreasoningapproachesbeenlargequestionrl-basedalgorithm
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recently, reinforcement learning (RL) has been shown to greatly enhance the reasoning capabilities of large language models (LLMs), and RL-based approaches have been progressively applied to visual multimodal tasks. However, the audio modality has largely been overlooked in these developments. Thus, we conduct a series of RL explorations in audio understanding and reasoning, specifically focusing on the audio question answering (AQA) task. We leverage the group relative policy optimization (GRPO) algorithm to Qwen2-Audio-7B-Instruct, and our experiments demonstrated state-of-the-art performance on the MMAU Test-mini benchmark, achieving an accuracy rate of 64.5%. The main findings in this technical report are as follows: 1) The GRPO algorithm can be effectively applied to large audio language models (LALMs), even when the model has only 8.2B parameters; 2) With only 38k post-training samples, RL significantly outperforms supervised fine-tuning (SFT), indicating that RL-based approaches can be effective without large datasets; 3) The explicit reasoning process has not shown significant benefits for AQA tasks, and how to efficiently utilize deep thinking remains an open question for further research; 4) LALMs still lag far behind humans auditory-language reasoning, suggesting that the RL-based approaches warrant further exploration. Our project is available at https://github.com/xiaomi-research/r1-aqa and https://huggingface.co/mispeech/r1-aqa.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VERITAS: A Multi-Agent Co-Scientist for Verifiable Image-Derived Hypothesis Testing

    cs.MA 2026-04 unverdicted novelty 7.0 of 10

    VERITAS is a multi-agent system for verifiable hypothesis testing on multimodal clinical MRI datasets that achieves 81.4% verdict accuracy with frontier models and introduces an epistemic evidence labeling framework.

  2. Weak-to-Strong On-Policy Distillation

    cs.LG 2026-07 conditional novelty 6.0 of 10

    A strong LLM is improved by distilling from the logit difference of two weaker models instead of from a stronger teacher.

  3. Audio-Zero: Label-Free Self-Evolution for Fine-Grained Audio Reasoning

    cs.SD 2026-07 conditional novelty 6.0 of 10

    A self-play game with a known 'odd listener' converts unlabeled audio contrast pairs into a verifiable reward, improving fine-grained audio reasoning on TREA, MMAU, and MMAR.

  4. Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos

    eess.AS 2026-07 conditional novelty 6.0 of 10

    AV-Flamingo is an open audio-visual language model that outperforms similarly sized open models on many long-video understanding benchmarks, using a new 7M-question dataset and timestamped reasoning.

  5. X$^3$-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment

    cs.LG 2026-07 conditional novelty 5.0 of 10

    X3-OPD improves audio-grounded reasoning by training the audio student on its own rollouts with token-level teacher feedback, using a three-tier paired text-audio corpus.

  6. AQA-TTRL: Self-Adaptation in Audio Question Answering with Test-Time Reinforcement Learning

    eess.AS 2025-10 conditional novelty 5.0 of 10

    Test-time reinforcement learning against self-generated majority-vote pseudo-labels improves audio question answering accuracy on MMAU, MMAR, and MMSU.

  7. Group Relative Policy Optimization for Speech Recognition

    eess.AS 2025-09 conditional novelty 5.0 of 10

    Applying GRPO with rule-based rewards to LLM-based ASR improves WER by up to 18.4% relative and reduces hallucination errors on unseen acoustic conditions.

  8. Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

    cs.CV 2025-08 unverdicted novelty 2.0 of 10

    A survey paper maps how external tools are used to augment multimodal large language models across data, tasks, evaluation, and future directions.

  9. Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models

    cs.CL 2025-05 conditional novelty 2.0 of 10

    A survey-style position paper claims that reinforcement fine-tuning powers reasoning in multimodal LLMs, summarizing over a hundred recent works and proposing five future research directions.

Pith tools