{"id":"0464ee12-4af3-402a-bbdd-c6f9060f3278","arxiv_id":"2607.20543","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"RLVR can destroy rare-but-recoverable correct trajectories on 'boundary' prompts; the PBA frozen-base anchor preserves them, improving both pass@1 and pass@256 over matched GRPO on Omni-MATH-Test.","lead":"Danger in RL training: improving first-try accuracy can erase rare correct answers that repeated sampling used to recover. The paper identifies the problematic 'boundary' problems and shows a simple prompt-level anchor (PBA) preserves them, improving both one-shot and many-shot performance.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Internal inconsistency in transition counts: Table 11 reports all base-solved→lost 201±17 while Table 12 reports boundary-only lost 654±35; boundary ⊂ all, so the central diagnostic numbers are irreconcilable as printed.","rationale":"The reader's verdict was CONDITIONAL, and the rationale already noted that 'Table 3 versus Table 11 transition counts (654 vs 201) are not reconciled.' I agree that the paper should not be accepted as-is, but I identify this count inconsistency as the single most load-bearing concern because it directly undermines the central quantitative claim rather than relying on sampling-noise arguments. The gate-noise concern is real (and the paper acknowledges it), but it is a matter of estimator precision that the authors partially address with Table 6 and explicit limitations. The count inconsistency is a logical contradiction in the reported evidence: if 'All base-solved→lost' is truly 201±17 and boundary-only is 654±35, the boundary attribution cannot be correct, and the abstract's '654±35 vs 91±18' numbers would be unsupported. If the 'All' row is instead a different metric, the paper needs a corrected table and a revised interpretation. Because this is checkable from the promised per-prompt artifact, the appropriate verdict remains CONDITIONAL: the paper's central diagnostic claim is conditionally supported pending reconciliation of the transition counts. I set agreement_with_reader to 'partial' because the reader's stated weakest assumption was the noisy G0=8 gate, whereas my primary concern is the table inconsistency (though the reader did flag it in the rationale). No ad hominem is intended; this is a request for data audit, not an accusation of fabrication.","tokens_in":12507,"tokens_out":6416,"duration_ms":67563,"concrete_test":"Run the artifact's scripts/summarize_seeds.py and scripts/bootstrap_passk.py on rollouts/eval_counts.parquet to recompute, for each seed and each regime, the four-cell base/trained transition matrices at pass@256 (and also at pass@1 as a definitional check). Then compare the 'All' row (sum over regimes) with the 'Boundary' row. If the artifact reproduces Tables 11/12 unchanged, the inconsistency is real; if the 'All' row was computed with a different 'base-solved' criterion, re-label it and re-state the aggregate loss. This single check determines whether the boundary-localization numbers in Table 3 survive.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline quantitative evidence for the boundary mechanism is internally inconsistent. In the 3000-prompt diagnostic study, Table 11 ('All base-solved→lost') reports 201±17 for GRPO and 73±13 for PBA, while Table 12 reports boundary-only 'base-solved→lost' as 654±35 for GRPO and 91±18 for PBA. Boundary prompts are a subset of all prompts, so the all-regime count cannot be smaller than the boundary count; the contradiction appears in every seed column (e.g., G0: 200 'all' vs 652 boundary). Either the 'All' row is mislabeled or computed with a different definition (e.g., base-solved at pass@1 rather than pass@256), or one of the two transition tables is wrong. The abstract and Sec. 1 rest the central claim on the 654 vs 91 boundary-loss numbers, so until this is reconciled, the claim that ordinary GRPO loses 7.2× more boundary prompts than PBA cannot be evaluated from the reported tables. This is not a judgment about intent; it is a request to audit the per-prompt counts that the artifact promises.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies pass@k inversion after RLVR: training can improve one-sample accuracy while reducing large-k coverage. It introduces a prompt-level diagnostic that classifies prompts by frozen-base behavior into solved-easy, boundary, reachable, and out-of-reach regimes, and argues that pass@k inversion is a boundary mode-commitment failure: boundary prompts contain rare correct trajectories that disappear under finite-sample GRPO before they are reinforced. It then proposes PBA, a prompt-conditional base anchor that gates sharpening on frozen-base rollout evidence and applies a KL anchor to risky prompts. Experiments on Qwen2.5-7B with GRPO on Omni-MATH-Test and MATH500 across three seeds report that ordinary GRPO improves pass@1 from 10.2 to 25.1 but leaves pass@256 at 68.3 (below base 69.1), while PBA reaches pass@1=29.0 and pass@256=73.0. A 3000-prompt diagnostic study claims GRPO loses 654±35 base-solvable boundary prompts versus 91±18 for PBA, a 7.2× reduction. The paper explicitly states that the collapse horizon is not a closed-form property of GRPO and that PBA is a proof-of-concept intervention.","tokens_in":12830,"tokens_out":7120,"duration_ms":70997,"significance":"If the empirical claims hold, the paper supplies a useful diagnostic framing for a phenomenon that pass@1-only evaluation misses: RLVR can shrink the reachable set even while sharpening the top sample. The promise of prompt-level, recoverability-gated regularization is timely, and the released artifact—per-prompt counts, seed-resolved tables, matched GRPO controls, gate ablations, and run cards—is a genuine strength. The two-mode model also yields a falsifiable prediction (boundary-localized coverage loss). However, the headline quantitative support is currently undercut by an internal inconsistency in the transition tables, and the central mechanistic claim is only an endpoint signature rather than a derivation. The significance is therefore conditional on the audit and on a recalibration of the claim's strength.","major_comments":[{"comment":"Table 11 reports 'All base-solved→lost' as 201±17 (seed G0=200) for GRPO, while Table 12 reports boundary-only 'Base-solved→lost' as 654±35 (seed G0=652). Boundary prompts are a subset of all prompts, so the all-regime count cannot be smaller than the boundary count. The same contradiction appears in every seed column. Since the 7.2× reduction in boundary lost prompts in Table 3 and Sec. 1 is the central empirical claim, this needs an audit: either the 'All' row uses a different definition of base-solved (e.g., pass@1) or one table is wrong. The per-prompt counts in the artifact should resolve this; as printed, the central diagnostic cannot be evaluated.","section":"Appendix B, Tables 11 and 12"},{"comment":"The paper's central claim is 'pass@k inversion is a boundary mode-commitment failure,' but Eq. (8) is assumed, not derived from GRPO, and Eq. (9) merely restates the sign of pass@k change conditional on that assumption. The text concedes the 'collapse horizon is not a closed-form property of GRPO' and the Limitations say there is no checkpoint-level validation of the collapse horizon. Consequently the experiments demonstrate boundary-localized loss, but not the proposed mechanism (commitment before rare positives are reinforced); global entropy collapse or other dynamics could produce the same endpoint. Please either soften the central claim to 'consistent with' mode commitment or provide checkpoint-level trajectory evidence.","section":"Sec. 5, Eq. (8), Proposition 1"},{"comment":"Regime labels and the PBA gate use finite frozen-base rollouts. With G0=8 and τ=0.10, a true w*=0.002, 0.01, or 0.05 has only 1.6%, 7.7%, or 33.7% probability of being detected (Table 1). The text says robustness to cutoffs is 'part of the required evidence,' but no sweep over the regime thresholds (solved/boundary/out-of-reach cutoffs) or over calibration-split noise is reported; Table 6 varies only G0 and τ. Please show that boundary lost counts and the PBA-vs-GRPO contrast are stable under these choices, or quantify the misclassification rates.","section":"Sec. 4 and Sec. 6, Table 1"}],"minor_comments":[{"comment":"'Boundary entropy↑' (Table 3) and 'Entropy retained' (Table 12) are used as mechanism evidence but no definition is given. Specify how entropy is measured (e.g., rollout-level token entropy, per-prompt answer diversity, or pass@k-based) before these numbers can be interpreted.","section":"Sec. 7, Tables 3 and 12"},{"comment":"The sampled token-level KL estimator can be negative for individual rollouts. The text notes that its expectation is the forward KL, but it would help to state explicitly whether the minibatch-average regularizer is used without per-rollout clipping, since clipping could bias the estimator.","section":"Sec. 6, Eq. (12)"},{"comment":"The 'Random protection' row is a single-mask control, and the paper correctly cautions against reading it as a distributional baseline. Consider labeling it as a demonstration rather than a control to avoid over-interpretation by readers.","section":"Table 5"},{"comment":"'Base-solved→lost' is used without a single formal definition. Define whether 'base-solved' means at least one verifier-positive completion in the frozen-base 256-sample cache, and define 'lost' as zero verifier-positive completions in the trained-policy evaluation cache. This would prevent ambiguity in the transition tables.","section":"Sec. 7 and Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The internal inconsistency between Tables 11 and 12 is serious enough that the paper should not be accepted in its current form. I am recommending major revision rather than reject because the issue is auditable from the artifact and the framing is useful. If the artifact confirms the counts as printed, the headline boundary-loss claim would be unsupported and I would move to reject."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, this paper gives a genuinely useful diagnostic for RLVR: it separates prompts into easy, boundary, reachable, and out-of-reach regimes, and shows how aggregate pass@k gains can hide a shrinking reachable set. Second, the central quantitative evidence as printed is internally inconsistent, and that undermines the headline claim until fixed.\n\nWhat is new: pass@k inversion itself was already reported by Yue et al. (and the paper cites them honestly). The genuinely new content is the boundary-regime decomposition, the finite-sample two-mode argument (Eqs. 5–9), and PBA, a simple prompt-conditional base anchor. The three-seed matched GRPO/PBA comparison is serious: full pass@k curves, per-prompt counts, seed CIs, and a set of single-run controls (global KL, entropy bonus, random protection, dropping, early stop, oracle boundary). That is real empirical work. The paper is also clear about its scope: it does not claim a calibrated collapse-horizon model, and it explicitly says the horizon is not a closed-form property of GRPO. Good honesty.\n\nThe soft spots. The load-bearing one is a table contradiction. Table 11 reports 'all base-solved→lost' for GRPO as 201±17; Table 12 reports boundary-only base-solved→lost as 654±35. Boundary prompts are a subset of all prompts, so the all-region count cannot be smaller than the boundary count. The same inversion appears in every seed column. Either one table is mislabeled or computed with a different definition, or one of them is wrong. Since the 654 vs 91 contrast drives the 7.2x claim, the authors need to audit their per-prompt counts and reconcile the definitions before that claim can be believed. The stress-test note is correct.\n\nSecond, the PBA gate uses G0=8 frozen-base rollouts, which is very noisy. Table 1 shows that for a true correct-mode mass of 0.002, the gate detects a positive only 1.6% of the time. The paper acknowledges this and even frames it as a feature (deliberately protect low-mass modes), but it means many prompts may be misclassified. The authors say robustness is part of the required evidence but don't show the cutoff sweep in the main text. This is a real gap, though not fatal.\n\nThird, no artifact URL or commit hash appears in the text, despite the reproducibility statement promising prompt-level records. For a workshop paper, that's a practical issue.\n\nWho this is for: anyone working on RLVR for reasoning models, evaluation methodology, or test-time compute. The diagnostic framing is already useful even if the specific numbers are fixed. I'd like to see the contradiction resolved and the gate-sensitivity analysis added. My own verdict: this deserves a serious referee; the empirical core is credible enough to warrant revision rather than rejection, but the transition counts need an audit and the tables must be reconciled. I would not cite it as-is; I'd wait for a corrected version.","headline":"A useful diagnostic paper with a clean seeded comparison, but the headline boundary-loss numbers are internally inconsistent across tables and need an audit before the main claim is credible.","tokens_in":13358,"tokens_out":3597,"would_cite":false,"duration_ms":36956,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Verifier-guided reinforcement learning can raise one-sample accuracy while erasing rare-but-recoverable correct answers, so pass@1 gains do not prove stronger reasoning.","keywords":["pass@k inversion","RLVR","GRPO","boundary prompts","mode commitment","base anchoring","coverage collapse","reasoning boundary"],"falsifier":"Use the paper's own 3000-prompt diagnostic split but recompute regime labels with 1024 frozen-base samples per prompt instead of 256; if ordinary GRPO still loses about 654 base-solvable boundary prompts, the localization holds, but if the loss drops toward PBA's 91, the inversion is an artifact of noisy base-rollout estimates. A second check: train on a constructed set of prompts with known correct-mode mass w and vary the RL group size G; inversion should track the probability (1-w)^G that a group misses the correct mode.","tokens_in":12261,"feed_emoji":"🧮","tokens_out":7354,"duration_ms":73014,"temperature":0.7,"pith_summary":"This paper tries to establish that pass@k inversion is a boundary-mode commitment failure: RL with verifiable rewards improves one-sample accuracy, but on prompts where the base model has rare correct trajectories recoverable by repeated sampling, finite training groups rarely see them and the policy commits to the wrong mode before they are reinforced. Because of this, ordinary training can solve fewer distinct problems at large k than the base model—on the olympiad-level benchmark, pass@256 drops from 69.1 to 68.3 even as pass@1 rises from 10.2 to 25.1. The paper demonstrates the failure localizes to boundary prompts (654±35 base-solvable prompts lost per seed) and that a prompt-conditional base anchor, PBA, preserves those prompts (91±18 lost) and lifts pass@256 to 73.0 while also improving pass@1. If correct, this reframes what RLVR gains mean: improvement on one sample cannot be read as stronger reasoning when the reachable set has shrunk. The practical stakes are large-k inference systems—self-consistency, verifier reranking, search, and agentic repair—whose value depends on rare recoverable modes surviving training.","feed_headline":"Training lifts one-shot accuracy but erases rare solvable answers","feed_subtitle":"A prompt-level gate preserves rare recoverable solutions, pushing many-attempt coverage above the untrained base.","key_machinery":"The central object is a two-mode model of each prompt as a mixture of a correct trajectory family with mass w*(x) and an incorrect family, so the one-sample base success probability equals w*(x). With an RL rollout group of size G, the probability a group contains no correct trajectory is (1 - w*)^G; for boundary prompts this is near 1, meaning updates provide no direct evidence for the correct mode before entropy falls and shared parameters drift. PBA, the Per-Problem Base Anchoring gate, is the operational probe: it estimates w*(x) from G0 frozen-base rollouts, applies ordinary GRPO when the estimate clears a threshold, and otherwise anchors the prompt to the frozen base via a token-level","core_discovery":"The paper's central claim is that pass@k inversion—verifier-guided reinforcement learning improving one-sample accuracy while reducing many-sample coverage—is a boundary mode-commitment failure, not generic entropy collapse. On a 3000-prompt diagnostic split, ordinary GRPO loses 654±35 prompts that the frozen base model could solve under repeated sampling, while leaving pass@256 (68.3) below the base's 69.1. A deliberately simple per-prompt base anchor, PBA, protects prompts whose base correct-mode mass is too low to show up in finite RL groups; it loses only 91±18 boundary prompts and lifts pass@256 to 73.0 while also improving pass@1 by about 3.9 points over matched GRPO. The paper argues","pith_inferences":["A natural extension is to replace the binary gate with a risk-weighted anchor whose strength scales as a lower confidence bound on the base correct-mode mass, which could protect more coverage at lower entropy cost than the one-success rule.","The same boundary-mode mechanism should appear in any repeated-attempt pipeline with automatic checkers—code repair, chart or spatial reasoning, tool-using agents—where a first-sample gain that erases rare recoverable trajectories would degrade test-time compute.","The theory predicts a quantitative stress test: vary the RL rollout group size G on prompts with known base correct-mass w; inversion should appear when G is small enough that (1-w)^G is close to 1 and disappear as G grows.","If the diagnosis holds, model evaluation for reasoning should move from single-sample accuracy to a coverage curve averaged over the base-to-trained transition; otherwise benchmark rankings can reverse under self-consistency or search."],"forward_implications":["If ordinary RLVR lifts pass@1 while pass@256 stays below the base model's, the improvement is not evidence of stronger reasoning: the set of reachable correct answers has likely shrunk.","Coverage loss should appear almost entirely on boundary prompts (base success below 10%, but many-sample recovery above 40%), leaving easy and out-of-reach prompts nearly unchanged—a signature that distinguishes this account from generic entropy collapse or support expansion.","Any verifier-guided training run should report full pass@k curves and base-to-trained transition counts, not just pass@1, because a single-number summary can hide destruction of rare recoverable modes.","A prompt-conditional trust region—sharpen where the frozen base shows enough correct evidence, anchor where it does not—can preserve high-budget coverage without sacrificing one-sample accuracy; PBA lifted both over matched GRPO.","Reasoning post-training should separate three operations: preserving rare useful modes, sharpening already-correct modes, and discovering new modes; ordinary outcome-reward RL without external exploration cannot do the third."],"fun_headline_variants":["RLVR boosts one-shot accuracy but kills many-sample coverage","Pass@k inversion: training that improves one-shot hurts repeated sampling","PBA gate preserves rare solvable prompts, lifts coverage above base","RLVR's pass@k inversion: rare wins lost for one-shot accuracy","Verifier RL loses rare solvable paths; per-prompt anchoring wins"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the frozen-base rollout counts used for both diagnosis and control—256 per prompt for regime labels and 8 for the PBA gate—are accurate enough to separate rare-but-recoverable boundary modes from out-of-reach noise; the paper itself concedes, in its limitations, that a true correct-mode mass of 1% is detected by the 8-rollout gate only 7.7% of the time, so misclassification in either direction could change the headline numbers.","fun_headline_variants_meta":{"raw":{"variants":["RLVR boosts one-shot accuracy but kills many-sample coverage","Pass@k inversion: training that improves one-shot hurts repeated sampling","PBA gate preserves rare solvable prompts, lifts coverage above base","RLVR's pass@k inversion: rare wins lost for one-shot accuracy","Verifier RL loses rare solvable paths; per-prompt anchoring wins"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001199,"raw_usage":{"total_tokens":4828,"prompt_tokens":844,"completion_tokens":3984,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":588,"completion_tokens_details":{"reasoning_tokens":3892}},"tokens_in":588,"tokens_out":3984,"duration_ms":30487,"temperature":1.0,"reasoning_tokens":3892,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T07:07:55.358695+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Use the paper's own 3000-prompt diagnostic split but recompute regime labels with 1024 frozen-base samples per prompt instead of 256; if ordinary GRPO still loses about 654 base-solvable boundary prompts, the localization holds, but if the loss drops toward PBA's 91, the inversion is an artifact of noisy base-rollout estimates. A second check: train on a constructed set of prompts with known correct-mode mass w and vary the RL group size G; inversion should track the probability (1-w)^G that a group misses the correct mode.","supporting_citations":[],"review_version":1}