{"id":"6872ba0a-7958-4e1a-b338-04fb46bb583f","arxiv_id":"2505.21178","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"ConciseR is a two-stage RL method that first improves reasoning accuracy with GRPO++ and then uses a length reward, gated on all rollouts being correct, to shorten chain-of-thought responses.","lead":"This paper trains a math reasoning model in two stages: first to be accurate, then to use shorter chain-of-thought text once it is accurate. The method, ConciseR, reports shorter responses with roughly unchanged accuracy on five competition math benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ConciseR's headline 'outperforms baselines' claim rests on a mixed-protocol pass@1 comparison; direct same-protocol re-evaluation with error bars is needed before accepting Table 3.","rationale":"The reader's verdict CONDITIONAL with concern about the heterogeneous pass@1 protocol is appropriate. My stress-test review of the paper text confirms the central claim rests on average accuracy advantages of 5.7 points (56.1 vs 50.4) against the best zero-RL baseline, with per-benchmark gaps of roughly 0-12 points. The evaluation section states the protocol explicitly for ConciseR (32 samples, sampling temperature 0.6, top-p 0.95) while the baseline table mixes results from reference [20] and other papers; no re-evaluation under a unified protocol is reported. Since pass@1 estimated from 32 sampled responses at T=0.6 is not numerically identical to greedy decoding, and since the paper provides no error bars or significance tests, the headline comparison is not yet established. The methodological contribution (two-stage RL with length reward gated on all-correct groups) is internally coherent: the reward design in Equation 7 only activates the length term when all group rollouts are correct, and the reported length reductions (20-23%) are consistent with the objective. Stage-1 GRPO++ vs Stage-2 L-GRPO results (55.3 vs 56.1 average) show only a small gain from the length optimization stage, but that stage's purpose is efficiency, so this is not a fatal internal inconsistency. The 'walk before you run' ablation claim is weakened by the absence of an ablation removing the all-correct gate, but the protocol comparability issue is the more load-bearing concern, since it determines whether the central 'outperforms' claim can be accepted at all. Agreement with the reader is full on the weakest assumption. Recommendation: keep CONDITIONAL; condition on unified-protocol re-evaluation and error bars. No ad hominem; no manufactured objections.","tokens_in":14291,"tokens_out":1736,"duration_ms":15673,"concrete_test":"Re-run ConciseR-Zero-7B and Oat-Zero-7B on AIME 2024, MATH-500, AMC 2023, Minerva, and OlympiadBench with the exact same protocol used for ConciseR (32 samples, temperature 0.6, top-p 0.95, max 3072 tokens, same prompt template, same answer extraction). If the average gap shrinks to within one standard error (bootstrapped over the benchmark questions), the 'outperforms' headline is unsupported. If the gap persists, the concern is resolved.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that ConciseR outperforms recent zero-RL reasoning models (Table 3). The load-bearing assumption is that the reported pass@1 numbers are comparable across baselines. The paper explicitly reports pass@1 with 32 samples at temperature 0.6 for ConciseR, while the cited baseline numbers (marked with †, from Hochlehnert et al. [20]) use heterogeneous decoding protocols: some are greedy, some use different sample counts and temperatures. Under a fixed protocol, a model's pass@1 can shift by several points; a 32-sample-then-average estimator at T=0.6 is not the same quantity as greedy decoding. The accuracy gaps ConciseR claims over Oat-Zero-7B (56.1 vs 50.4 average), the strongest zero-RL baseline, are on the order of a few points per benchmark and could be protocol artifacts. The paper provides no error bars or significance tests, so the reported stability of accuracy across L-GRPO steps (Figure 3) is not evidence against this. The load-bearing weakness is therefore the comparability of the pass@1 numbers, not the internal consistency of the two-stage RL algorithm.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ConciseR, a two-stage reinforcement learning framework for training math reasoning models (starting from Qwen2.5-Math-7B) to produce shorter chain-of-thought responses without sacrificing accuracy. The first stage trains reasoning ability via GRPO++ (GRPO with clip-higher, dynamic sampling, and an entropy bonus); the second stage applies L-GRPO, which adds a length reward that is active only when all sampled rollouts for a training question are correct. The authors report that ConciseR outperforms recent zero-RL reasoning models on AIME 2024, MATH-500, AMC 2023, Minerva, and OlympiadBench while reducing response length by roughly 20%.","tokens_in":14734,"tokens_out":7681,"duration_ms":75628,"significance":"If the claims hold, the work is practically valuable: a roughly 20% inference-cost reduction at approximately constant accuracy would meaningfully lower deployment costs for long-CoT reasoning models. The proposed gating heuristic ('walk before you run') is simple and interesting, and the paper honestly documents a failure mode of a direct length-reward variant. The significance is currently limited by the absence of same-protocol baseline comparisons and the lack of any uncertainty quantification, both of which are required to support the headline 'outperforms' claim.","major_comments":[{"comment":"The pass@1 numbers for ConciseR are computed with 32 samples at temperature 0.6, while the baseline numbers marked with a dagger are taken from Hochlehnert et al. [20] and were produced under heterogeneous decoding protocols (varying sample counts, temperatures, or greedy decoding). Pass@1 is sensitive to the sampling protocol, so the few-point gaps over Oat-Zero-7B (50.4 vs 56.1 average) do not by themselves establish that ConciseR outperforms these baselines. Please re-evaluate at least the strongest zero-RL baselines under the exact ConciseR protocol, or provide protocol-matched numbers from the original sources.","section":"Sec. 4 'Evaluation Setup' and Table 3"},{"comment":"No error bars, confidence intervals, or multiple seeds are reported for any accuracy estimate. AIME 2024 has only 30 questions, so a 3-5 point difference can easily arise from sampling noise; the same concern applies to Minerva and OlympiadBench. The claim that accuracy remains stable across L-GRPO steps is not supported without quantifying the sampling variability. Please provide bootstrap confidence intervals for the pass@1 estimates or repeat the evaluation under additional random seeds.","section":"Sec. 4.1 and Figure 3"},{"comment":"The all-correct gating condition is the paper's central design principle, but it is not ablated. The failure-experience experiments in Section 5.3 compare L-GRPO against two alternative length-reward formulations, but they do not test whether the gating on all-correct rollouts is necessary or superior to, for example, applying the length reward unconditionally or to groups with at least one correct answer. Without such an ablation, the claim that this gating is what preserves accuracy is unsupported.","section":"Sec. 3.2 and Sec. 5.3"},{"comment":"The abstract states that ConciseR 'generates more concise CoT reasoning responses,' but no response-length numbers for any baseline model are reported. The length reductions of 21-23% cited in Section 4.1 are relative to ConciseR's own first-stage model. To substantiate the comparative conciseness claim, please report average output lengths for the baselines under the same evaluation protocol.","section":"Abstract and Sec. 4.1"}],"minor_comments":[{"comment":"The benchmark name 'OlympaidBench' should be spelled 'OlympiadBench'.","section":"Sec. 4 'Evaluation Benchmarks'"},{"comment":"The sentence 'The used training template is shown in Figure 2' should refer to Table 2, which actually contains the template.","section":"Sec. 4 'Evaluation Setup'"},{"comment":"The text says 14,022 responses correspond to 1,558 questions times 11 test times, but 1,558 times 11 is 17,138; Figure 6 uses 15,580 for the same product. Please correct the arithmetic so the counts match.","section":"Sec. 5.1 and Figure captions"},{"comment":"The claim of an 'average accuracy improvement of 55.2%' compared to the base model is unclear: Table 3 shows 37.5 to 56.1, which is a 49.6% relative improvement or an 18.6-point absolute gain. Please specify the formula used.","section":"Sec. 4.1"},{"comment":"The conclusion says ConciseR 'significantly outperforming existing efficient reasoning methods' without the zero-RL qualifier, which is too broad given that FastCuRL-1.5B-V3 and DeepScaleR-1.5B-Preview in Table 3 have higher or comparable averages. Please qualify the claim.","section":"Sec. 7 Conclusion"},{"comment":"Equation (8) uses the constraint '0 < |{oi|is_equivalent(oi,a)}|' while Equation (6) uses '0 < ... < G'; please explain whether all-correct groups are intentionally included in L-GRPO updates and align the notation.","section":"Eq. (8) vs. Eq. (6)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a fairly direct extension of DAPO and FastCuRL, with the two-stage schedule and the all-correct length-reward gate as the main novel components. The evaluation protocol mismatch is a serious but fixable problem: re-running the strongest baselines under the authors' own protocol would either confirm or overturn the headline claim. The paper promises open release of code, data, and checkpoints, which is a strength if fulfilled. The lack of uncertainty quantification is also fixable and should be addressed in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. The core recipe is a two-stage RL schedule: first train reasoning with GRPO++ (DAPO-style clip-higher, dynamic sampling, entropy bonus), then train conciseness with a length reward that only activates when all rollouts for a sample are correct. That gating rule (Eq. 7) and the walk-before-you-run schedule are the genuinely new bits, and they are sensible. The paper also reports honest failure experiments: directly rewarding shorter responses caused the model to skip reasoning and guess, so they switched to rewarding unused context length. That kind of candid reporting is useful.\n\nWhat the paper does well: the length reductions (~20-23% across five benchmarks) are consistent with the objective, and Figure 3 shows accuracy roughly flat while length drops. The method is simple, reproducible in principle (code, data, checkpoints promised), and the problem—overthinking in long-CoT models—is real and important.\n\nThe soft spots are in the evaluation, and they are load-bearing. The headline 'outperforms' rests on Table 3, where ConciseR's scores come from 32 samples at temperature 0.6, while most baseline numbers are taken from other papers (marked † from Hochlehnert et al.) with heterogeneous decoding protocols (greedy or different sample counts). Pass@1 at T=0.6 with 32 samples is not the same quantity as greedy pass@1; it can shift by several points. The claimed margins over Oat-Zero-7B (56.1 vs 50.4 average) are a few points per benchmark, so protocol differences could explain them entirely. There are also no error bars, seeds, or significance tests, even on the 30-question AIME where a 0.5-point difference is noise. A directly comparable baseline—same base model, same data, same eval protocol, with and without the length gate—is missing.\n\nNone of this undermines the internal story: the second stage cuts length without hurting accuracy, and the stage-1 vs stage-2 comparison is fair since both use the same protocol. But the 'outperforms prior zero-RL methods' claim is not yet supported.\n\nMy take: this is a solid workshop-to-conference paper with a useful, concrete idea and an honest write-up. It deserves a serious referee, but the referee should ask for a re-evaluation of baselines under a matched protocol with error bars before the central claim is accepted. I'd read it in a journal club and would cite the gating rule as related work, but I wouldn't yet trust the accuracy comparison.","headline":"Two-stage RL recipe with an all-correct-gated length reward is a sensible, concrete contribution, but the 'outperforms baselines' claim rests on unmatched evaluation protocols and no error bars.","tokens_in":15088,"tokens_out":2814,"would_cite":true,"duration_ms":27448,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-stage reinforcement learning schedule that first trains reasoning ability and only then, when every sampled response in a group is correct, optimizes for shorter chain-of-thought produces more concise responses and beats zero-RL…","keywords":["concise reasoning","chain-of-thought","reinforcement learning","GRPO","length reward","overthinking","LLM reasoning","zero RL paradigm"],"falsifier":"Run ConciseR-Zero-7B and each zero-RL baseline in Table 3 through one identical evaluation script with the same 32-sample temperature-0.6 pass@1 protocol; if the average accuracy gap over Oat-Zero-7B (56.1 vs 50.4) closes or reverses, the central outperformance claim fails. Separately, retrain stage 2 with the length reward applied unconditionally; if accuracy falls, the all-correct gate is doing the work.","tokens_in":14111,"feed_emoji":"⚡","tokens_out":6689,"duration_ms":66872,"temperature":0.7,"pith_summary":"The paper tries to establish that a two-stage reinforcement learning schedule can make a large language model reason more concisely without losing accuracy. The first stage strengthens the model's problem-solving ability through GRPO++ (GRPO with clip-higher, dynamic sampling, and an entropy bonus); the second stage applies a length-aware reward that only activates when every sampled response for a question is correct. Trained this way, ConciseR-Zero-7B produces noticeably shorter chain-of-thought responses and, in the paper's comparison, outperforms existing zero-RL reasoning baselines across five competition-level math benchmarks. The point matters because long chain-of-thought responses are expensive to generate and often contain redundant 'overthinking', so a method that cuts length while keeping accuracy has a direct computational payoff.","feed_headline":"Two-stage RL trims LLM reasoning by 20% without losing accuracy","feed_subtitle":"Only after all sampled answers are correct does the model get a length reward—accuracy holds while responses shrink.","key_machinery":"The load-bearing object is the length-aware reward used in L-GRPO: for a group of $G$ responses to one question, the reshaped reward is $\\hat r_i = r_i + \\lambda \\hat L_i$ with $\\hat L_i = 1 - L_i/L_{\\mathrm{Max}}$ when $\\sum_i r_i = G$ (all rollouts correct) and $\\hat L_i = 0$ otherwise. This keeps the standard binary accuracy reward for the group-relative advantage computation and adds a gentle pressure to finish within fewer tokens, but only after the group has already solved the problem. The 'walk before you run' gate is what distinguishes L-GRPO from naive length penalties: it prevents the policy from shortening responses by giving up on hard reasoning, the failure mode the paper documents in its ablation of direct length rewards. GRPO++ supplies the first-stage machinery—asymmetric clip-higher bounds, dynamic sampling, and an entropy bonus—that lets the model acquire the reasoning ability before compression begins.","core_discovery":"The central claim is that concise reasoning can be trained without trading away accuracy. Starting from Qwen2.5-Math-7B, the paper first runs GRPO++ to incentivize correct problem solving, then switches to L-GRPO, which reshapes the group-relative reward to include a length term $\\hat L_i = 1 - L_i/L_{\\mathrm{Max}}$ only in groups where all rollouts are correct. This gating embodies the 'walk before you run' principle: the model is never rewarded for brevity until it has already achieved correctness on that example. On AIME 2024, MATH-500, AMC 2023, Minerva, and Olympiad, the resulting model averages 56.1% pass@1 (k=32), above the best zero-RL baseline's 50.4%, while response lengths fall roughly 20% to 23% across the five benchmarks. The paper also reports that the first stage alone already surpasses the zero-RL baselines, suggesting the compression stage adds efficiency on top of a stronger reasoning model.","pith_inferences":["The gating idea should transfer to other verifiable efficiency targets—e.g., fewer tool calls, lower latency, or shorter proofs—whenever a group of rollouts can be certified correct.","The observed shift toward Python-based verification during stage 2 suggests that compression pressure may push models toward more reliable external checks; this is a pattern the paper displays, not a claim it establishes.","Because the comparison mixes evaluation protocols, the compression effect (a within-paper, matched-protocol result) is more robust than the ranking claim; a matched-protocol rerun could plausibly preserve the length reduction but reorder the baselines."],"forward_implications":["Under the paper's evaluation protocol, ConciseR-Zero-7B scores 56.1 average pass@1 across AIME 2024, MATH-500, AMC 2023, Minerva, and Olympiad, above the best zero-RL baseline's 50.4.","Response lengths drop by about 20% to 23% across the five benchmarks after stage 2, while per-benchmark accuracy stays flat or improves.","The all-correct gate prevents the length reward from making the model abandon reasoning: in the paper's ablations, direct length rewards caused empty reasoning and answer guessing.","GRPO++ on its own already beats all zero-RL baselines, so the conciseness stage layers efficiency onto a model that has first been made stronger."],"supporting_citations":[{"why":"Supplies the zero-RL GRPO training paradigm and the R1-distilled baselines whose overthinking motivates the work.","marker":"[5]"},{"why":"Provides the base GRPO objective that both stages modify.","marker":"[6]"},{"why":"Contributes clip-higher and dynamic sampling, adopted as GRPO++, along with the DAPO training set.","marker":"[12]"},{"why":"Provides the DeepScaleR dataset mix used for stage-1 training and a 1.5B Preview baseline.","marker":"[8]"},{"why":"FastCuRL is the closest efficient-reasoning baseline whose Preview and V3 scores ConciseR must beat.","marker":"[11]"},{"why":"Oat-Zero-7B is the strongest zero-RL baseline in Table 3 (50.4 average) that ConciseR claims to outperform (56.1).","marker":"[13]"},{"why":"Qwen2.5-Math-7B is the base model all ConciseR runs start from.","marker":"[22]"},{"why":"Source of the dagger-marked baseline numbers in Table 3, making the comparison protocol visible.","marker":"[20]"}],"fun_headline_variants":["Two-stage RL: shorter thinking, same accuracy","RL brevity trick: only reward conciseness after correct answers","Shorter CoT, same score: new RL strategy","Concise reasoning trained with RL, accuracy intact","Walk before run: RL concise reasoning without loss"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central comparison assumes that the paper's pass@1 scores, computed with 32 sampled responses at temperature 0.6, are numerically comparable to baseline numbers in Table 3 that come from other papers using possibly different decoding protocols (greedy or different sample counts); if that comparability fails, the claim that ConciseR outperforms the baselines loses its support.","fun_headline_variants_meta":{"raw":{"variants":["Two-stage RL: shorter thinking, same accuracy","RL brevity trick: only reward conciseness after correct answers","Shorter CoT, same score: new RL strategy","Concise reasoning trained with RL, accuracy intact","Walk before run: RL concise reasoning without loss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00023,"raw_usage":{"total_tokens":1518,"prompt_tokens":1019,"completion_tokens":499,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":635,"completion_tokens_details":{"reasoning_tokens":422}},"tokens_in":635,"tokens_out":499,"duration_ms":5141,"temperature":1.0,"reasoning_tokens":422,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:33:01.845907+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run ConciseR-Zero-7B and each zero-RL baseline in Table 3 through one identical evaluation script with the same 32-sample temperature-0.6 pass@1 protocol; if the average accuracy gap over Oat-Zero-7B (56.1 vs 50.4) closes or reverses, the central outperformance claim fails. Separately, retrain stage 2 with the length reward applied unconditionally; if accuracy falls, the all-correct gate is doing the work.","supporting_citations":[{"cited_title":"Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica","cited_arxiv_id":null,"evidence_quote":"Provides the DeepScaleR dataset mix used for stage-1 training and a 1.5B Preview baseline."},{"cited_title":"Fastcurl: Curriculum reinforcement learning with progressive context extension for efficient training r1-like reasoning models, 2025","cited_arxiv_id":null,"evidence_quote":"FastCuRL is the closest efficient-reasoning baseline whose Preview and V3 scores ConciseR must beat."},{"cited_title":"A sober look at progress in language model reasoning: Pitfalls and paths to reproducibility, 2025","cited_arxiv_id":null,"evidence_quote":"Source of the dagger-marked baseline numbers in Table 3, making the comparison protocol visible."}],"review_version":1}