{"id":"80e9a141-cf8e-4850-99fc-8dc9e01aac54","arxiv_id":"2412.13292","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"RAD iteratively refines a distribution over answers by marginalizing over refinement samples, improving accuracy on six arithmetic benchmarks over self-consistency and hint-based prompting.","lead":"This paper introduces RAD, a method that improves language model reasoning by repeatedly refining the distribution over answers rather than individual answers. It reports accuracy gains over self-consistency and hint-based prompting on math word problems at equal sampling budgets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The stated flow condition guarantees only one RAD step; the iterative claim that pr(y|x) increases with r is not established, and the paper's p3-vs-p1 check does not test it.","rationale":"The reader's CONDITIONAL verdict is fair. The empirical evaluation is broad—36 scenarios, multiple LLMs, multiple benchmarks—and the reported gains over CoT+SC and PHP+SC are credible, with statistical tests and a detailed difficult-question analysis. The concern I identify does not refute those empirical results; rather, it shows that the theoretical story for the iterative component is incomplete. Even if the Section 3.1 flow condition holds for one step, the transition from p1 to p2, the paper's claim that pr(y|x) increases with r does not follow from the stated assumptions, because the required condition becomes stricter as pr(y|x) grows. The empirical evidence compares p3 with p1, not consecutive distributions, so it is compatible with a non-monotone trajectory (e.g., p1 < p2 but p3 < p2). This is not fatal: RAD may still be a useful algorithm, and the majority of tested configurations show accuracy gains. But the paper should either prove a multi-step condition or present consecutive-iteration comparisons and direct kernel estimates. I therefore keep the reader's CONDITIONAL verdict unchanged. I partially agree with the reader's identified weakest assumption: we both point to the probability flow condition, but my concern sharpens it to the missing multi-step guarantee and the inadequacy of the p3-vs-p1 test for that guarantee.","tokens_in":25316,"tokens_out":10738,"duration_ms":105369,"concrete_test":"On a fixed benchmark (e.g., GSM8K with GPT-3.5 Turbo), rerun CoT+RAD while recording the full empirical distributions p1, p2, p3 for every question. Separately estimate the refinement kernel q(y)=p(y|x,Refine(y)) and the inflow terms p(y|x,Refine(y')) by sampling with oracle or known wrong hints on a held-out split. For each r=1,2, test whether p_{r+1}(y) > p_r(y) and whether q(y) exceeds the iteration-specific threshold 1 - [Σ_{y'≠y} p_r(y') p(y|x,Refine(y'))]/p_r(y). If monotonicity or the threshold fails on a substantial fraction of questions, the theoretical claim should be revised to a one-step result rather than an iterative guarantee.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's theoretical claim (Section 3.1, Eq. 1) is that the probability of the true answer grows with r. The condition stated there—p(y|x,Refine(y)) > 1 - c(1-p1(y|x)) with inflow p(y|x,Refine(y')) ≥ c p1(y|x)—is sufficient only for the first update. At iteration r, the same inflow bound gives p_{r+1}(y)-p_r(y) ≥ c p1(y)(1-p_r(y)) - p_r(y)(1-q(y)), where q(y)=p(y|x,Refine(y)). For p_{r+1}(y) ≥ p_r(y), one needs q(y) ≥ 1 - c p1(y)(1-p_r(y))/p_r(y). As p_r(y) grows, this threshold becomes stricter. Example: with p1(y)=0.4, c=0.3, q(y)=0.82, the stated one-step condition holds (threshold 0.82), but at p2(y)=0.56 the required q is about 0.906; the same kernel would decrease mass at the next iteration. Thus the assertion that repeated application is further advantageous is not justified by the paper's condition. The empirical support (Tables 6-7, Figures 4-5) compares p3 with p1, not consecutive distributions, so it cannot establish monotonicity across iterations. If q(y) is not very close to 1, extra RAD rounds can erode the gain from the first round, which bears directly on the choice of R=3 and on the method's core mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Refined Answer Distributions (RAD), an iterative refinement framework in which a distribution over candidate answers is updated by marginalizing over previous-round answers: p_{r+1}(y|x) = ∫ p(y|x, Refine(y')) p_r(y'|x) dy' (Eq. 1). The authors give an intuitive one-step 'probability flow' argument in Section 3.1, a Monte Carlo implementation in Section 3.2 and Appendix A, and an extensive empirical evaluation on six arithmetic benchmarks, MATH, and two BIG-Bench Hard tasks with five LLMs. RAD variants are reported to achieve the highest accuracy in 30 of 36 scenarios, and additional analyses (Figures 3-5, Tables 6-7) are used to support the claim that RAD increases the probability of the correct answer relative to initialization.","tokens_in":25666,"tokens_out":8647,"duration_ms":75357,"significance":"If the empirical finding holds, RAD is a useful and simple addition to the toolbox for LLM reasoning: it reuses the refinement prompt of Progressive Hint Prompting but maintains a distribution and marginalizes over previous answers, which is a clean and budget-matched way to go beyond self-consistency. The experimental breadth is a genuine strength: five models, several benchmarks, a controlled LLM-call budget, and Wilcoxon tests on the top-two comparisons. The main weakness is that the theoretical narrative in Section 3.1 is not established for multiple iterations, and the empirical verification of the governing condition is indirect. Since the central contribution is primarily empirical, the paper is salvageable with a corrected or weakened theoretical claim and a direct test of the underlying condition.","major_comments":[{"comment":"The probability-flow condition stated in Section 3.1 is sufficient for a single update only, and the text's assertion that 'repeated application of this procedure is further advantageous' is not justified by that condition. Let q(y)=p(y|x,Refine(y)), and suppose p(y|x,Refine(y')) ≥ c p1(y|x) for all y'≠y. For iteration r, the lower bound is p_{r+1}(y) ≥ q(y)p_r(y) + c p1(y|x)(1-p_r(y)), so p_{r+1}(y) ≥ p_r(y) requires q(y) ≥ 1 - c p1(y|x)(1-p_r(y))/p_r(y). As p_r(y) grows, the required q becomes stricter; with the paper's own values p1(y)=0.4 and c=0.3, the one-step threshold is 0.82, but if the first step yields p2(y)=0.56 (as in the paper's illustration), the same kernel would need q(y) ≥ 0.906 for the second step to increase mass. The empirical comparison in Tables 6-7 is between p3(y|x) and p1(y|x), not between consecutive distributions, so it cannot establish monotonicity across iterations. The authors should either prove a condition that is uniform in r, weaken the theoretical claim to one update, or report comparisons of pr+1 with pr for each r; this matters directly because the default algorithm runs R=3 iterations.","section":"Section 3.1, Eq. (1)"},{"comment":"The claim that the hint-based prompt 'satisfies the probability flow criterion' is not verified by the evidence presented. The criterion is a statement about the conditional refinement distributions q(y)=p(y|x,Refine(y)) and p(y|x,Refine(y')) for incorrect y', but Tables 6-7 and Figures 4-5 report p3(y|x) versus p1(y|x), which are the very quantities the method is intended to increase. This is an indirect check at best: a large p3-vs-p1 gain could occur even if the per-step condition fails, and a small gain could occur even if the condition holds. A direct test would estimate q(y) and the inflow probabilities on held-out questions (or at least on a separate partition) and check the inequality of Section 3.1 pointwise or in aggregate. Please either provide such a test or explicitly re-label the current analysis as evidence about the final outcome rather than about the flow condition.","section":"Section 3.2 (Discussion) and Appendix E, Tables 6-7"},{"comment":"The recursion in Eq. (1) implicitly assumes a Markov property: the distribution of the refined answer depends only on the previous answer y' and not on the earlier history, such as whether the same hint has already been shown in a previous round or what chain of thought preceded it. This assumption is not stated, and the implementation in Algorithm 1 samples from p(·|x,Refine(ym)) using only the immediately previous answer. If history matters, the Monte Carlo estimator in Eqs. (4)-(5) is not an unbiased estimator of the marginal pr+1 defined by the true refinement process. The authors should state this assumption explicitly and, if possible, test it, for example by comparing the outcome distribution of a second-round refinement given the same hint y' with and without an intermediate different hint.","section":"Section 3.2, Eq. (1) and Algorithm 1"}],"minor_comments":[{"comment":"The table header 'Llama-3-70b-intruct' contains a typo and should read 'Llama-3-70b-instruct'.","section":"Table 4 (Appendix C)"},{"comment":"The mode assignment in line 14 is placed inside the r loop and refers to p_R even though R is the total number of iterations; move the mode assignment after the loop or index it by r.","section":"Algorithm 1 (Appendix A)"},{"comment":"The caption says the test is 'for the difficult questions (for the entire dataset)', but each cell contains two p-values; please explain in the caption what the parenthetical values represent.","section":"Table 6 caption"},{"comment":"The phrase '3'rd decimal rounding off' should be 'third decimal rounding'; also, the statement that RAD has 'comparable token cost' would be easier to assess if actual token counts or API costs were reported in the main experiments as they are for the MATH dataset.","section":"Section 4, experimental setting"}],"recommendation":"major_revision","confidential_remarks":"The paper is a workshop-style contribution with a strong empirical core. The theoretical claim in Section 3.1 is the main obstacle to acceptance: it is stated more strongly than what is proved, and the empirical verification of the flow condition is indirect. I would encourage the editor to ask for a direct test of the flow condition and a revision of the theoretical language. I also did not find a statement about code or data release; making the evaluation code available would considerably increase the value of this work for a journal audience."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about arXiv:2412.13292. First, the empirical claim is real: RAD improves reasoning accuracy over CoT+SC and PHP+SC across a wide set of benchmarks and models, with a fair budget comparison and statistical tests. Second, the theoretical justification in Section 3.1 is weaker than the paper suggests—the stated probability-flow condition suffices only for one refinement step, not for the repeated iterations that the algorithm uses.\n\nThe genuinely new idea here is maintaining a full distribution over answers and updating it by weighted marginalization over refinement samples. That is not present in the prior self-consistency or hinting work. The experiments are broad: 36 scenarios, five models, arithmetic plus MATH and two Big-Bench Hard tasks, with Wilcoxon signed-rank tests on the main comparisons. The 'difficult questions' analysis is a nice touch that supports the claim that RAD helps exactly where sampling-based methods struggle. They were also careful about matching LLM-call budgets with PHP+SC. Given the same budget, RAD wins 30 of 36 scenarios. That is a solid empirical contribution.\n\nThe soft spot is the theory. The probability-flow condition in Section 3.1, p(y|x,Refine(y)) > 1 - c(1-p1(y|x)) with inflow bounded by c p1, proves p2(y) > p1(y). It does not prove p3 > p2 or monotonicity generally. The stress-test example is correct: as p_r(y) grows, the required q(y) gets steeper, and a kernel that passes the one-step test can decrease mass at the next iteration. The paper asserts repeated application is 'further advantageous' and runs R=3, but the stated condition does not guarantee it. The empirical p3-vs-p1 comparison in Tables 6-7 is not the same as checking consecutive iterations; it can hide a non-monotone path where the second round erodes part of the first round's gain.\n\nAlso minor: the flow condition is verified only on the same benchmarks used for the accuracy claim, which limits the strength of that evidence. No code is released, and there is no sensitivity analysis for the budget split (B1, B2, B3) or the constant c.\n\nBottom line: the method works empirically and the distribution-refinement formulation is worth attention. The theory needs to be corrected or explicitly weakened to a one-step guarantee, and the authors should either prove the iteration bound or replace the assertion with empirical evidence on consecutive distributions. This deserves a proper referee, not a desk reject. I would cite it for the method, not the theory.","headline":"Empirically useful distribution-refinement method with a theory section that overclaims: the flow condition proves one step, not monotonicity across iterations.","tokens_in":26179,"tokens_out":2991,"would_cite":true,"duration_ms":27687,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By refining the entire distribution over answers—feeding each previous answer back to the LLM as a hint and weighting the responses by their prior probability—RAD shifts probability mass toward the correct answer and beats…","keywords":["refined answer distributions","iterative refinement","self-consistency","chain-of-thought prompting","probability flow condition","hint-based prompting","LLM reasoning","Monte Carlo answer distribution"],"falsifier":"Compute, on a held-out reasoning benchmark, the empirical refinement probabilities $p(\\tilde{y}=y\\mid x,\\mathrm{Refine}(y))$ and $p(\\tilde{y}=y\\mid x,\\mathrm{Refine}(y'))$ for true answers $y$ and wrong hints $y'$; if the first is not close to 1 for a substantial fraction of questions, or the second falls below the constant-fraction lower bound, then the paper's predicted monotone rise in correct-answer probability will fail on exactly those questions, and a majority of such failures would refute the central claim.","tokens_in":25115,"feed_emoji":"🧠","tokens_out":9480,"duration_ms":80019,"temperature":0.7,"pith_summary":"The paper presents Refined Answer Distributions (RAD), a way to improve LLM reasoning that refines the whole distribution of answers rather than individual chains of thought. Each round feeds every distinct answer from the previous round back to the LLM as a hint and then marginalizes the refined responses, weighting each by the probability that the hinted answer was correct. The paper argues that if a 'probability flow' condition holds, the probability of the correct answer grows with each round, so the mode of the refined distribution is more likely to be right. On arithmetic, MATH, and BIG-Bench Hard tasks spanning five LLMs, RAD variants are reported to achieve the highest accuracy in 30 of 36 scenarios compared with self-consistency, progressive-hint prompting, and their combinations, using the same number of LLM calls.","feed_headline":"Refining answer distributions beats self-consistency in 30 of 36 tests","feed_subtitle":"Weighting refinements by prior answer probabilities puts more mass on the true answer at the same sampling budget.","key_machinery":"The load-bearing object is the sequence of refined answer distributions $p_r(\\tilde{y}|x)$ together with the marginalization formula in Eq. (1), implemented as a weighted Monte Carlo update. At iteration $r$, the empirical distribution is represented as weighted point masses on the distinct answers seen so far; each distinct answer $y_m$ is inserted into the prompt as a hint, a budget of refinement samples is drawn from $p(\\tilde{y}|x,\\mathrm{Refine}(y_m))$, and the new distribution is obtained by pooling those samples with weights $\\omega_m$ equal to the estimated probability of $y_m$ under $p_r$. The named 'probability flow' condition is what gives the update its power: it guarantees that more probability flows into the correct answer from refinements of wrong answers than flows out of it, so the mode can move toward the truth over rounds. The implementation uses the hint-based prompting of Zheng et al. (2023) as the refinement kernel.","core_discovery":"The central claim is that iterating the update $p_{r+1}(\\tilde{y}|x)=\\int p(\\tilde{y}|x,\\mathrm{Refine}(y'))\\,p_r(y'|x)\\,dy'$ concentrates probability mass on the true answer $y$ as $r$ grows, provided the LLM's refinement behavior satisfies the 'probability flow' condition of Section 3.1. Informally, handing the model the correct answer as a hint must keep it correct with high probability, and handing it a wrong answer must still leave a constant fraction of the chance of recovering the correct answer; under that condition the inflow of mass to $y$ from refinements of wrong answers exceeds the outflow from refining $y$ itself. Because the correct-answer probability rises, the mode of $p_r$ is a better estimator than the mode of the original chain-of-thought distribution, and the final answer is the mode of a Monte Carlo approximation to $p_R$. The experimental section reports that this mechanism holds broadly: out of 36 model and dataset scenarios, the RAD variants attain the highest accuracy in 30.","pith_inferences":["Reading Eq. (1) as a linear map, $p(\\tilde{y}|x,\\mathrm{Refine}(y'))$ is a stochastic kernel over answers, so RAD is iterating a Markov chain on the answer space; Markov-chain convergence rates and stationary-mode conditions could be imported to predict when refinement will help.","The 'probability flow' condition is a property of the LLM-plus-prompt pair, not of the answer set, so one could screen candidate refinement prompts on a labeled validation set before deployment and only run RAD when the condition is empirically satisfied.","The paper allocates refinement samples uniformly across distinct previous answers; concentrating samples on high-weight answers would likely reduce the variance of the mode estimate, since the Monte Carlo error is dominated by answers with large $\\omega_m$."],"forward_implications":["If RAD is correct, the self-consistency plateau is not a sampling-budget limit but a distributional one: later LLM calls spent on refining distinct answers redistribute probability mass and raise accuracy beyond what 40-sample voting achieves.","RAD can be initialized from CoT or from progressive-hint prompting, so it is an orthogonal layer that can be stacked on other prompting techniques without redesigning their prompts.","With the same LLM-call budget, RAD improves accuracy on hard questions: on GSM8K with GPT-3.5 Turbo, CoT+RAD reaches 87.5% versus 83.2% for CoT+SC.","On the MATH dataset, PHP+RAD improves accuracy in 5 of 7 subdisciplines over CoT+SC while costing about 2.9 cents per question versus 2.5 cents for CoT+SC.","For 'difficult' questions across the arithmetic benchmarks, RAD more often assigns the highest probability to the true answer than CoT+SC or PHP+SC, and the increase from $p_1(y|x)$ to $p_3(y|x)$ is statistically significant in 31 of 36 comparisons."],"supporting_citations":[{"why":"Supplies the self-consistency baseline and majority-vote setting that RAD must exceed at equal sample budgets.","marker":"Wang et al. 2023"},{"why":"Supplies the progressive-hint-prompting refinement kernel used in RAD's implementation and a baseline competitor.","marker":"Zheng et al. 2023"},{"why":"Supplies the few-shot chain-of-thought prompts that initialize the answer distribution and define the CoT baseline.","marker":"Wei et al. 2022b"},{"why":"Documents the plateau of self-consistency gains after a handful of samples, which motivates the iterative redistribution strategy.","marker":"Aggarwal et al. 2023"},{"why":"Supplies the fair-evaluation finding that many refinement and self-correction methods fail to beat self-consistency, framing RAD's comparison.","marker":"Huang et al. 2024"},{"why":"Provides the GSM8K benchmark used for the main arithmetic evaluation and the detailed probability-flow analyses.","marker":"Cobbe et al. 2021"},{"why":"Provides the MATH benchmark used to compare RAD against multi-agent prompting at comparable token cost.","marker":"Hendrycks et al. 2021"}],"fun_headline_variants":["RAD beats self-consistency in 30/36 reasoning tests","Refining answer distributions: 30/36 wins for LLM reasoning","Iterative answer refinement beats self-consistency in 30/36","New algorithm refines answer distributions, wins 30/36 tests"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's convergence claim rests on the unproven 'probability flow' condition: when the LLM is prompted with the correct answer as a hint it must answer correctly almost always, and when prompted with a wrong answer it must still produce the correct answer with probability at least a constant times the initial correct-answer probability; if that condition gives way, the recursive update can concentrate mass on a wrong answer.","fun_headline_variants_meta":{"raw":{"variants":["RAD beats self-consistency in 30/36 reasoning tests","Refining answer distributions: 30/36 wins for LLM reasoning","Iterative answer refinement beats self-consistency in 30/36","New algorithm refines answer distributions, wins 30/36 tests"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000916,"raw_usage":{"total_tokens":3912,"prompt_tokens":908,"completion_tokens":3004,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":2926}},"tokens_in":524,"tokens_out":3004,"duration_ms":21023,"temperature":1.0,"reasoning_tokens":2926,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:15:12.311085+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute, on a held-out reasoning benchmark, the empirical refinement probabilities $p(\\tilde{y}=y\\mid x,\\mathrm{Refine}(y))$ and $p(\\tilde{y}=y\\mid x,\\mathrm{Refine}(y'))$ for true answers $y$ and wrong hints $y'$; if the first is not close to 1 for a substantial fraction of questions, or the second falls below the constant-fraction lower bound, then the paper's predicted monotone rise in correct-answer probability will fail on exactly those questions, and a majority of such failures would refute the central claim.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the progressive-hint-prompting refinement kernel used in RAD's implementation and a baseline competitor."}],"review_version":1}