{"id":"5e679f53-761a-4cfa-a703-ce5e71c2a5f2","arxiv_id":"2504.15900","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A curriculum-guided reinforcement learning recipe with structured chain-of-thought improves audio question answering, reaching 67.08% on MMAU test-mini.","lead":"This paper trains an audio AI model to think out loud before answering multiple-choice sound questions, using supervised examples followed by reinforcement learning ordered from easy to hard. The best model reaches 67.08% on the MMAU audio benchmark, but the result needs scrutiny because training and test data overlap and no code or data is released.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline accuracy rests on an unvalidated LLM judge; exact-match rescoring could erase or invert the structured-CoT advantage.","rationale":"The reader's weakest assumption is train/test overlap on MMAU. That overlap is real and acknowledged in Section 4.2, but domain overlap alone does not necessarily invalidate the comparison if the test questions are original; a deduplication analysis would clarify it. I identify a more fundamental and more easily testable threat: the unvalidated Qwen2-72B-Instruct judge used for all MMAU accuracy numbers. Because the paper's central contribution is that structured reasoning helps, and because structured outputs are verbose and well-organized, the judge could systematically favor the structured-CoT models regardless of the option chosen. This would directly undermine the 67.08% SOTA claim and the structured-vs-unstructured comparisons. The concern is concrete and falsifiable: exact-match rescoring or human-rated subsets would settle it. I still credit the paper's controlled ablations, the clear demonstration that SFT warm-up is necessary for stable RL, and the internal consistency of most qualitative findings. These strengths, plus the fact that the judge concern remains unproven, keep the verdict at CONDITIONAL rather than moving to REJECT. The condition should be validation of the evaluation metric or public release of model outputs for independent rescoring.","tokens_in":13447,"tokens_out":3739,"duration_ms":37821,"concrete_test":"Obtain the authors' model outputs on MMAU test-mini (or regenerate with a released harness) and rescore them two ways: (1) exact/rule-based extraction of the letter or text in <ANSWER> compared to the gold option; (2) a 100-item subset scored by the same Qwen2-72B-Instruct judge and by two human annotators, reporting Cohen's kappa. Recompute all deltas in Tables 1-3. If exact-match accuracy differs from judge accuracy by more than 2 points, or if the structured-vs-unstructured gap (e.g., 67.08 vs 65.90) shrinks below the sampling noise of 4 trials, the SOTA and structured-reasoning conclusions are not supported by the reported evaluation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2 defines MMAU accuracy via a Qwen2-72B-Instruct judge that 'jointly analyzes' the question, reference answer, and model output, but provides no validation of this judge. Every headline number in Tables 1-3 depends on it: the 67.08% SOTA, the 16.35% gain, and the structured-vs-unstructured deltas (67.08 vs 65.90 on Qwen2.5-Omni; 65.55 vs 63.68 on Qwen2-Audio). Only 4 responses per model are averaged, with no variance or test-retest statistics reported. A soft judge is especially dangerous here because structured CoT outputs are verbose and formatted, so the judge could reward format or reasoning fluency rather than the selected option; the paper's central claim that structured reasoning is superior would then be an evaluation artifact. The reward during RL is exact answer correctness, while evaluation uses a different, unvalidated metric, creating a train/eval mismatch. No inter-annotator agreement, human correlation, or comparison to exact option matching is reported, despite the text asserting the evaluation is 'reliable and robust.' Independently of the acknowledged train/test domain overlap, this unvalidated judge is the most load-bearing weakness: it determines whether the main comparisons are real.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends GRPO-based reinforcement learning to large audio-language models (LALMs). The authors construct a 32k-sample multiple-choice audio QA corpus from AudioSet, MusicBench, MELD, and AVQA, generate structured and unstructured chain-of-thought annotations with Qwen2.5-72B, and train via SFT followed by curriculum-guided GRPO. They report that the resulting SARI model, built on Qwen2.5-Omni, achieves 67.08% average accuracy on the MMAU test-mini benchmark, which they describe as state-of-the-art, and a 16.35-point improvement over the Qwen2-Audio-7B-Instruct base model. Ablations on Qwen2-Audio-7B-Instruct compare implicit vs. explicit reasoning, structured vs. unstructured CoT, and with vs. without curriculum learning, concluding that SFT warm-up is necessary, structured CoT generalizes better, and easy-to-hard curricula improve final performance.","tokens_in":13639,"tokens_out":5599,"duration_ms":48373,"significance":"If the reported numbers withstand scrutiny, this is a useful contribution: it is one of the first systematic studies of GRPO with curriculum learning in the audio modality, and the controlled comparisons of reasoning styles under identical architectures are informative. The authors also report negative results (RL from scratch fails to induce meaningful CoT), which adds credibility. However, the central quantitative claims rest on an unvalidated LLM judge and a training/test domain overlap that is acknowledged but not analyzed. The claimed SOTA and the structured-vs-unstructured advantage are therefore not yet established to the standard required for publication.","major_comments":[{"comment":"All headline numbers in Tables 1–3 depend on the Qwen2-72B-Instruct judge that 'jointly analyzes' the question, reference answer, and model outputs, yet no validation of this judge is provided: no agreement with exact-match extraction, no human correlation, and no comparison with the official MMAU protocol. This is load-bearing because the RL reward is exact option correctness, while the reported metric is a different, soft, unvalidated judgement. The risk is that the judge rewards verbose, formatted structured-CoT outputs irrespective of the actual selected option, which would inflate the structured-vs-unstructured deltas (67.08 vs. 65.90 on Qwen2.5-Omni; 65.55 vs. 63.68 on Qwen2-Audio). Please report exact-match accuracy on the extracted final answer, agreement statistics (e.g., Cohen's kappa) between the LLM judge and exact match, or adopt the official MMAU scoring procedure.","section":"§4.2 (Evaluation Metrics)"},{"comment":"The text acknowledges that MMAU test-mini 'evaluates both in-domain performance on tasks similar to our training data (since we use some AudioSet and music questions)', but it does not provide any overlap or deduplication analysis between the training corpus (AudioSet, MusicBench, MELD, AVQA) and the test set. If test clips or question templates appear in the training data, the SOTA claim (67.08%) and the 16.35-point improvement could reflect memorization or in-domain fit rather than reasoning ability. Please provide a deduplication analysis (audio clip identity, question n-gram overlap, or source-level breakdown) and, if overlap exists, report results on the non-overlapping subset separately.","section":"§4.2 (Train/test overlap)"},{"comment":"The paper reports a single average over four sampled responses per model, with no per-response variance, standard errors, or significance tests. The structured-vs-unstructured differences (e.g., 67.08 vs. 65.90 on Qwen2.5-Omni; 63.58 vs. 63.65 for model-d vs. model-e) are within plausible sampling noise, yet they are used to support the central claim that structured reasoning is superior. Please provide per-response results, bootstrap confidence intervals, or a paired significance test over the four samples, and state the number of test items in each subset.","section":"Tables 1–3 and §4.2 (No variance/significance)"}],"minor_comments":[{"comment":"The phrase '16.35% improvement in average accuracy' is an absolute difference in percentage points (65.55 vs. 49.20), not a relative percentage improvement; please use '16.35 percentage points' or the relative figure (≈33%).","section":"Abstract and §5"},{"comment":"Please correct typos: 'Howover' (§2), 'as showed' (§3.1), 'outperformes' (§5), 'LL AVA-COT' (§3.1), 'V oiceBench' (ref. [32]), and 'T able' (§5).","section":"Throughout"},{"comment":"The claim of 'robust generalization capabilities' based on MMSU is overstated for the Qwen2-Audio-7B-Instruct variant: SARI's MMSU accuracy (35.36) is slightly below the base model (35.72). Please qualify the claim or restrict it to the Qwen2.5-Omni variant, which does improve MMSU from 61.32 to 66.01.","section":"§5 (Generalization claim)"},{"comment":"Please specify whether the Qwen2.5-Omni pipeline uses the same curriculum ordering (computed from Qwen2-Audio-7B pass rates) or a model-specific ordering, since this affects reproducibility.","section":"§4.1 and Tables 2–3"},{"comment":"The KL divergence penalty is disabled (coefficient set to 0), which is unusual for GRPO; please discuss the rationale and any observed effect on training stability.","section":"§3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper's acceptance hinges on the validity of the evaluation metric and the degree of train/test overlap. I strongly recommend that the editor require the authors to provide exact-match agreement statistics for the LLM judge and a deduplication/overlap analysis. If the judge is not validated and the overlap is not quantified, the SOTA and structured-CoT advantage claims should be withdrawn or heavily qualified. The core methodology appears sound, but the reported numbers are not yet trustworthy enough for archival publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Zhi, quick take on SARI. The paper is a solid engineering study with a real question behind it; the headline numbers, though, are softer than the abstract suggests. The core claim — that SFT warm-up plus curriculum-guided GRPO makes audio-language models reason better when they output a structured chain of thought — is plausible and the ablations are mostly coherent. That part deserves engagement.\n\nWhat's actually new: they adapt GRPO to audio QA with a curriculum schedule, build a 32k MCQ corpus, and systematically compare implicit vs explicit, structured vs unstructured reasoning under matched architectures. The SFT-warm-up result is well demonstrated: from-scratch RL (models b/c) produces degenerate CoT and no gain, while SFT+RL (d/e) does much better. That is a useful, reproducible-sounding finding.\n\nThe soft spots are in the evaluation, and the stress-tester is right that the unvalidated Qwen2-72B judge is the load-bearing issue. Every headline number — the 67.08% SOTA, the 16.35% gain, the structured-vs-unstructured deltas — comes from that judge, not from exact option matching. No validation, no human correlation, no inter-annotator agreement. Because structured CoT is verbose and formatted, a soft judge could easily reward fluency rather than the chosen option. The RL reward is exact correctness while the eval metric is something else, so the comparisons could measure an artifact. That is a serious concern, not a nit.\n\nSecond, Section 4.2 admits the MMAU test-mini overlaps with their training domains (AudioSet, music) but gives no dedup or contamination analysis. So the 'state-of-the-art' is an in-domain claim at most. Third, the curriculum benefit on unstructured reasoning is +0.03 points — negligible — yet the conclusion says curriculum 'substantially enhances' performance. That is overstated; the benefit is only clear for structured CoT. Missing code/data also limits reproducibility, but that's fixable.\n\nBottom line: this is a promising recipe paper, not a proven result. Send it to a serious referee who demands judge validation and exact-match rescoring; without that, the SOTA claim should not stand. I'd cite it as an existence proof of the recipe, but not as evidence for the magnitude.","headline":"Structured CoT + curriculum GRPO is a plausible recipe for audio reasoning, but the unvalidated LLM judge makes the headline SOTA numbers provisional until exact-match rescoring.","tokens_in":14224,"tokens_out":1817,"would_cite":false,"duration_ms":16893,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a curriculum-guided reinforcement-learning recipe, built on structured chain-of-thought supervision, raises audio question-answering accuracy by 16.35% over the base model and reaches 67.08% on the MMAU test-mini…","keywords":["reinforcement learning","GRPO","audio-language model","chain-of-thought","curriculum learning","structured reasoning","MMAU benchmark","multiple-choice audio QA"],"falsifier":"Take the MMAU test-mini items, split them into clips that overlap the 32k training set and clips that do not, and recompute accuracy on each slice; if the non-overlapping slice shows no improvement over the base model, the reported gain is memorization, not reasoning. A second check is to train with the same recipe but evaluate on a benchmark with no domain overlap and no multiple-choice cues.","tokens_in":13169,"feed_emoji":"🎧","tokens_out":4750,"duration_ms":39110,"temperature":0.7,"pith_summary":"This paper asks whether reinforcement learning, which sharpens reasoning in text-only language models, can sharpen reasoning in models that listen. It answers yes: by first fine-tuning an audio-language model on chain-of-thought answers, then running group-relative policy optimization (GRPO) on a 32,000-sample multiple-choice audio corpus ordered from easy to hard, the model improves by 16.35 percentage points in average accuracy over its base and reaches 67.08% on the MMAU test-mini benchmark. The authors report that structured reasoning outperforms free-form reasoning and that supervised warm-up is necessary for stable RL training. If the benchmark numbers hold up, the recipe would give audio assistants a way to reason audibly and transparently about what they hear.","feed_headline":"Structured RL thinking lifts audio QA accuracy by 16.35%","feed_subtitle":"A curriculum-trained audio model hits 67.08% on MMAU test-mini while keeping its reasoning trace readable.","key_machinery":"The engine is the GRPO reinforcement-learning update applied to a large audio-language model after a supervised warm-up. The structured reasoning format, divided into Planning, Caption, Reasoning, and Summary sections, forces the model to write down what it hears before judging it. The curriculum orders the 30,000-sample RL pool by the base model's pass rate, from easiest to hardest, so the policy collects reliable rewards early. The reward itself is simple—correct answer plus format adherence—and the KL penalty is disabled.","core_discovery":"The central claim is that explicit, structured reasoning plus curriculum-guided GRPO transfers the 'think before answering' paradigm to large audio-language models. The best model, SARI, built on Qwen2.5-Omni, scores 67.08% average accuracy on MMAU test-mini, which the paper labels state-of-the-art, and 66.01% on the cross-domain MMSU set; the Qwen2-Audio-based variant beats its base by 16.35 percentage points. The ablations attribute the gain to three ingredients: supervised fine-tuning on structured chains as a warm start, GRPO with answer-correctness reward, and an easy-to-hard curriculum. The authors also report that structured chains generalize more robustly than unstructured ones and that curriculum learning accelerates convergence.","pith_inferences":["A likely next test is whether the same recipe works for open-ended audio reasoning (no multiple-choice options) and with a reward that grades reasoning quality rather than only answer correctness.","The structured format's Caption step may be the key transferable mechanism: writing down what is heard before reasoning could be adapted to other modalities, such as video or sensor streams, where grounding is the bottleneck.","Because the dataset is built entirely from open-source captions and LLM-generated questions, the recipe is reproducible at low cost; one testable extension is whether larger or more capable caption generators further raise the ceiling."],"forward_implications":["Supervised warm-up is a prerequisite: RL from scratch on the raw audio model produces meaningless chains and no accuracy gain, so the two-stage recipe is the transferable ingredient.","Structured chains beat unstructured ones on the same architecture, suggesting that forcing an explicit Caption step reduces errors that come from missing audio cues.","Curriculum ordering matters as much as the reward: easy-to-hard GRPO converges faster and lands at a higher final accuracy than the same training on a shuffled set.","The gains carry over to a larger base model (Qwen2.5-Omni), where the structured variant reaches 67.08% on MMAU test-mini and 66.01% on MMSU, indicating the recipe scales with base capability."],"supporting_citations":[{"why":"Supplies the GRPO algorithm used for the RL stage, replacing a critic network with group-relative rewards.","marker":"[7]"},{"why":"Defines the base model Qwen2-Audio-7B-Instruct that all ablation variants start from.","marker":"[11]"},{"why":"Provides the structured chain-of-thought format (Planning/Caption/Reasoning/Summary) and the prior Audio-Reasoner baseline.","marker":"[12]"},{"why":"Earlier attempt at GRPO on audio QA that found limited gains; this work extends it with SFT warm-up and curriculum.","marker":"[13]"},{"why":"Introduces curriculum learning, the easy-to-hard schedule used to order the RL training samples.","marker":"[14]"},{"why":"Defines the MMAU benchmark whose test-mini set is the main evaluation and the source of the state-of-the-art claim.","marker":"[15]"},{"why":"Defines the MMSU/MMLU-Pro-based set used to test cross-domain generalization.","marker":"[32]"}],"fun_headline_variants":["Structured audio reasoning via RL reaches 67.08% on MMAU test-mini","Curriculum-guided RL sharpens audio LLMs to 67.08% on MMAU","Think before listening: structured reasoning lifts audio QA by 16.35%","Audio reasoning gets a structured RL boost: +16.35% on accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on MMAU test-mini being a fair measure of generalization, but the training corpus overlaps with the benchmark's domain (AudioSet and music questions), and no deduplication or overlap analysis is shown.","fun_headline_variants_meta":{"raw":{"variants":["Structured audio reasoning via RL reaches 67.08% on MMAU test-mini","Curriculum-guided RL sharpens audio LLMs to 67.08% on MMAU","Think before listening: structured reasoning lifts audio QA by 16.35%","Audio reasoning gets a structured RL boost: +16.35% on accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000821,"raw_usage":{"total_tokens":3602,"prompt_tokens":960,"completion_tokens":2642,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":2554}},"tokens_in":576,"tokens_out":2642,"duration_ms":19319,"temperature":1.0,"reasoning_tokens":2554,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:14:42.535825+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the MMAU test-mini items, split them into clips that overlap the 32k training set and clips that do not, and recompute accuracy on each slice; if the non-overlapping slice shows no improvement over the base model, the reported gain is memorization, not reasoning. A second check is to train with the same recipe but evaluate on a benchmark with no domain overlap and no multiple-choice cues.","supporting_citations":[],"review_version":1}