{"id":"39b6f4c9-c605-4292-b18f-fc6f8ac8ed02","arxiv_id":"2502.01618","paper_version":5,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Particle-filtering selection of LLM reasoning paths scales inference-time compute 4 to 16 times more efficiently than beam search on math tasks and lets small models match much larger closed models.","lead":"This paper uses a sampling technique called particle filtering to keep a diverse set of candidate solutions while a language model reasons step by step, instead of pruning early guesses. On math benchmarks, this lets small open models match or beat GPT-4o and o1-level accuracy with very few rollouts, pointing to a cheaper way to scale reasoning performance.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1's per-step softmax resampling over cumulative product rewards does not implement the SMC weight recursion, so Theorem 1's unbiasedness is unsupported and the claimed probabilistic-inference basis for the empirical gains is not established.","rationale":"The paper's empirical results could survive even if Theorem 1 is invalid, so I am not moving to REJECT. But the paper explicitly motivates the method as a 'principled application of probabilistic inference tools' and claims PF 'provides consistent, unbiased estimators under mild assumptions'. The mismatch between Algorithm 1 and the standard SMC weight recursion is the least secure load-bearing condition for that claim. I agree with the reader's weakest_assumption, and the conditional verdict is appropriate: either repair the proof or reframe the method as stochastic beam search, and add error bars or multiple seeds for the headline scaling numbers. A toy-SMC check would settle the theoretical question directly.","tokens_in":23488,"tokens_out":8390,"duration_ms":77065,"concrete_test":"Run Algorithm 1 exactly as specified on a two-step toy state-space model with known transition p, known reward r, and known correctness function; compute the weighted estimate sum_i w^{(i)} is_correct(x^{(i)}) over many independent runs and compare its mean to E_{p_hat(x_{1:2}|o_{1:2}=1)}[is_correct]. If the means differ beyond Monte Carlo error, Theorem 1 is false as stated. A cheaper check: instrument the code to print pre-resampling weights; if w is the cumulative product and resampling occurs every step, the effective target is r_1^2 r_2 rather than r_1 r_2, contradicting Eq. (2).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is that Algorithm 1 is a valid sequential Monte Carlo sampler for the posterior in Eq. (2), so that Theorem 1's unbiasedness holds. As written, Algorithm 1 updates w = [rhat(x_{1:t}^{(i)})] and then resamples with theta = softmax(w) at every step, while Section 3.1 states the weight recursion w_t ∝ w_{t-1} * rhat(c, x_{<t}). These are not the same update. If rhat(x_{1:t}) is the default product-of-step-rewards cumulative score, resampling every step with the cumulative product double-counts earlier rewards: after t resamplings the effective target is p(x_{1:T}) * prod_{s=1}^T r_s^{T-s+1}, not p(x_{1:T}) * prod_s r_s. If rhat(x_{1:t}) is instead only the current step's reward, the cumulative likelihood in Eq. (2) is never formed. Either way, the proof in Appendix C, which is a two-sentence appeal to 'the unbiasedness property of particle filtering' and is labeled as Theorem 2 while the main text calls it Theorem 1, does not establish the claimed result. The same issue propagates to the Particle Gibbs and parallel-tempering extensions, whose acceptance probabilities assume a correct PF target. This does not by itself disprove the empirical 4-16x scaling claim, but it removes the 'principled probabilistic inference' basis that is a stated core contribution, and it means the method is currently better described as stochastic beam search with a softmax proposal.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes to treat inference-time scaling of LLMs as probabilistic inference in a state-space model, using particle filtering (PF) to sample from a posterior over reasoning trajectories defined by an LLM transition model and a PRM-based emission model. The authors present Algorithm 1 (PF), extensions to Particle Gibbs and parallel tempering, and an extensive empirical evaluation on MATH500, AIME 2024, FinanceBench, and NumGLUE. They claim a 4-16x better scaling rate than deterministic search baselines, and that small open models such as Qwen2.5-Math-1.5B-Instruct can surpass GPT-4o with only 4 rollouts, while Qwen2.5-Math-7B-Instruct reaches o1-level accuracy with 32 rollouts. A theorem is stated asserting unbiased estimation of expected accuracy from the particle weights, with a proof in Appendix C.","tokens_in":23785,"tokens_out":7935,"duration_ms":65954,"significance":"If the empirical results hold, the paper would make a valuable practical contribution: inference-time scaling with stochastic resampling instead of deterministic beam-style pruning is an appealing and simple idea, and the breadth of experiments across model families and non-math benchmarks is a strength. The claimed 4-16x compute savings would be practically important, and the connection to sequential Monte Carlo could open a useful design space. However, the manuscript's theoretical backbone is central to its framing, and that backbone is not currently sound: the stated algorithm does not implement the weight recursion that the theorem relies on, and the proof is only a citation-level assertion. The empirical claims also lack error bars and are partly based on test-set hyperparameter selection. The paper deserves a major revision to either fix the theoretical mismatch, or to reframe the contribution as a stochastic search heuristic with strong empirical results and remove the unsubstantiated unbiasedness guarantees.","major_comments":[{"comment":"Algorithm 1 updates the weight vector as w = [rhat(x^{(i)}_{1:t})] and resamples with probabilities proportional to exp(w_i) at every step, whereas §3.1 specifies the multiplicative weight recursion w^{(i)}_t ∝ w^{(i)}_{t-1} · rhat(c, x^{(i)}_{<t}). These updates are inconsistent. If rhat(x_{1:t}) denotes the cumulative product of per-step rewards, resampling with that cumulative product at every step double-counts earlier rewards, yielding an effective target proportional to p_M(x_{1:T}|c) ∏_{s=1}^T r_s^{T-s+1} rather than Eq. (2); if rhat(x_{1:t}) denotes only the current step's reward, the cumulative likelihood in Eq. (2) is never formed. The proof of Theorem 1/Theorem 2 in Appendix C is a two-sentence appeal to 'the unbiasedness property of particle filtering' and does not establish that this property holds for the algorithm actually implemented. Because the unbiasedness theorem is the stated basis for calling the method probabilistic inference rather than stochastic beam search, this issue is load-bearing. The authors should either modify Algorithm 1 to implement a valid SMC weight recursion (e.g., update via incremental likelihood factors and resample only when the effective sample size is low), or replace the theorem and the 'principled probabilistic inference' claims with an honest description of the method as a stochastic search heuristic.","section":"§3.1 / Appendix E, Algorithm 1"},{"comment":"The main results on MATH500 (Table 1, Figure 4) are obtained after selecting the process reward model, temperature, reward aggregation strategy, and particle budget using 100-question subsets of MATH500 (Section 4.5). If those 100 questions are part of the 500 problems used for the reported accuracy, the headline numbers reflect test-set selection. The paper should state explicitly whether the validation questions were excluded from the final evaluation and, ideally, choose hyperparameters on a separate held-out split or report sensitivity across a range of hyperparameter choices.","section":"§4.5, §4.2"},{"comment":"The central scaling claim ('4-16x better scaling rate') and the model-comparison claims (e.g., Qwen2.5-Math-7B PF 87.7% vs o1-preview 87.0% on MATH500; 6/30 vs 4/30 on AIME 2024) are reported without error bars, confidence intervals, or any statistical significance assessment, as acknowledged in the NeurIPS checklist. On AIME 2024 with 30 problems, the 2-problem difference between PF and beam search is within binomial noise; on MATH500 a 0.7-point difference is likewise not established as significant. The authors should provide per-seed or bootstrap variability for the main comparisons, or explicitly temper the claims (e.g., 'matches' instead of 'surpasses' where differences are within noise).","section":"§4.4, Table 1"},{"comment":"The compute budget is not defined consistently. Algorithm 1 generates one new step per particle at each time step, so over a full rollout the number of policy-model calls is N × T, not N, yet Figure 4 and the text treat the particle count N as the 'budget (# of model generations)' and compare against beam search/DVTS under the same label. If the baselines count one generation as one complete solution, while PF counts one particle as one complete solution, the total number of decoding calls is larger for PF at the same nominal budget. The paper should define 'generation' precisely and ensure the budget axis measures the same resource for all methods, including the cost of the PRM and ORM.","section":"§3.1 / Figure 4"}],"minor_comments":[{"comment":"The theorem is labeled 'Theorem 1' in the main text but 'Theorem 2' in Appendix C, and the paragraph before it refers to 'Theorem 2 (proof in Appendix C)'. The numbering and cross-references should be made consistent.","section":"§3.1 / Appendix C"},{"comment":"Equation (1) writes the emission as p(ot | c, x_{<t}) but defines it as B(ot; r(c, xt)); the observation should depend on xt, not x_{<t}. Also, the notation x_{<t−1} in Section 2 and Eq. (1) is confusing; it should be x_{<t} or x_{1:t-1}.","section":"§2, Eq. (1)"},{"comment":"In Algorithms 2 and 3, the transition step is written as 'x^{(i)}_{t+1} ~ pM(· | c, x^{(i)}_{t+1})' which conditions on the variable being generated; this should presumably be pM(· | c, x^{(i)}_{1:t}).","section":"Appendix E, Algorithms 2 and 3"},{"comment":"The sentence 'While formulating it as a MAB problem allows it to use a scheduling on the softmax temperature and to derive regret bounds, we no longer have the same unbiasedness from the particle filtering / SMC formulation' is confusing and appears to compare the wrong pair of methods; it should be clarified or rephrased.","section":"§5, Related Work"},{"comment":"The caption for Table 2 is missing; the table title currently reads 'Method FinanceBench NumGLUE Task 2 (Chemistry)' as a heading. The caption and formatting should be fixed.","section":"§4.3 / Table 2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has a strong and broad empirical component, and the core idea of using stochastic resampling instead of deterministic pruning is worth publishing if properly situated. However, the theoretical claims are currently not supported by the presented algorithm, and the empirical evaluation has two concerns that could affect the reliability of the headline numbers: absence of error bars/significance testing, and hyperparameter selection on subsets of the same test sets. The paper is likely fixable within its scope by (a) making the algorithm-theory correspondence explicit or dropping the unbiasedness theorem, (b) adding variability estimates or softening claims, and (c) clarifying the budget definition and validation splits. I recommend major revision rather than rejection because the empirical results appear consistent and the methodological idea is interesting, but the paper must not be accepted in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the quick take: the empirical recipe is interesting — softmax-resampling a particle population instead of top-k pruning gives real gains on math reasoning across model families — but the paper's own algorithm doesn't do what the theory says it does. The scaling results are worth taking seriously; the probabilistic framing as written is not.\n\nWhat's new: applying particle-based resampling to inference-time scaling with an off-the-shelf PRM, no training, with a careful empirical comparison against beam search, DVTS, and BoN/WBoN. The 4-16x scaling-rate claim is bold, and the ablations on reward aggregation, temperature, and PRM choice are exactly what you'd want. The extension to Particle Gibbs and parallel tempering is a reasonable bonus, even if the gains there are modest. They also cite the SMC-for-LLM literature fairly, which sets their contribution apart from the earlier trained-verifier approaches.\n\nThe soft spots are real, and the biggest one is the theory. Algorithm 1 sets w = rhat(x_{1:t}) at every step and resamples from a softmax of those cumulative scores. That is not the SMC weight recursion w_t ∝ w_{t-1} rhat(c, x_{<t}) described in Section 3.1, and it's not the target posterior in Eq. (2). The result is a stochastic beam search with a softmax proposal, not a particle filter. The proof of Theorem 1 is two sentences and just appeals to 'the unbiasedness property of particle filtering'; it doesn't derive anything from the actual algorithm. (Also, the theorem is called Theorem 1 in the main text and Theorem 2 in the appendix — sloppy but minor.) If the framing were fixed to be honest about what the algorithm does, the empirical contribution would stand on its own.\n\nTwo more issues, in proportion. No error bars; the authors admit this, and the headline numbers like '1.5B surpasses GPT-4o in 4 rollouts' are single runs. And key hyperparameters — PRM, temperature, budget — were selected on a 100-problem subset of MATH500, which is test-set tuning. That doesn't make the results fake, but it inflates the apparent gains relative to baselines that don't get that tuning.\n\nBottom line: this is a legitimate and potentially useful empirical method, but the theoretical packaging is currently unsupported. The authors should either repair the weight recursion so the algorithm matches the claimed target, or reframe as stochastic beam search and drop the unbiasedness claim. Either way it deserves a serious referee; with error bars and an honest framing it could be a solid paper.\n\nRecommendation: send it to review, but expect the theory section to need a major overhaul.","headline":"Particle-resampled beam search is a genuinely promising scaling trick, but the paper's SMC theory does not match its own Algorithm 1; worth reviewing as an empirical method.","tokens_in":24376,"tokens_out":3093,"would_cite":true,"duration_ms":27480,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that replacing deterministic top-k search with softmax-sampling particle filtering gives inference-time scaling a 4–16x better scaling rate, letting small open models match much larger closed models on math reasoning.","keywords":["inference-time scaling","particle filtering","sequential Monte Carlo","process reward models","large language models","mathematical reasoning","beam search","state-space models"],"falsifier":"Run Algorithm 1 on a small toy state-space model with enumerable states and compare the weighted-particle estimate of any expectation, such as expected accuracy, against the exact value computed from $\\hat{p}_M(x_{1:T} \\mid c, o_{1:T}=1)$; if the estimate stays biased as the number of particles grows, the unbiasedness theorem fails and the method is a stochastic search heuristic. A cheaper check is to inspect whether the implementation multiplies incremental likelihoods $w^{(i)}_t \\propto w^{(i)}_{t-1} \\hat{r}(c, x^{(i)}_{\\le t})$ or instead replaces the weight with the raw cumulative PRM score, since the pseudo-code in Algorithm 1 does the latter and that breaks the standard SMC recursion.","tokens_in":23246,"feed_emoji":"🎲","tokens_out":9609,"duration_ms":71679,"temperature":0.7,"pith_summary":"The paper claims that inference-time scaling of large language models should be treated as probabilistic inference rather than search, and that carrying this out with particle filtering makes test-time compute far more sample-efficient. Where beam search and dynamic variable-time search greedily prune low-scoring partial solutions and can never recover them, the proposed method keeps a weighted population of candidate trajectories and resamples them stochastically, so promising paths are favored but not allowed to dominate. On math and broader reasoning benchmarks the authors report a 4–16x better scaling rate than deterministic search counterparts, and show that Qwen2.5-Math-1.5B-Instruct surpasses GPT-4o accuracy with only 4 rollouts while Qwen2.5-Math-7B-Instruct reaches o1-level accuracy with 32. If this holds, small open models can close much of the gap with proprietary frontier models purely by spending inference compute more intelligently.","feed_headline":"Particle filtering scales LLM reasoning 4–16x faster than beam search","feed_subtitle":"A 1.5B math model beats GPT-4o with only 4 rollouts; a 7B model reaches o1-level accuracy at 32.","key_machinery":"The load-bearing machinery is particle filtering (sequential Monte Carlo) applied to a state-space model whose transition kernel is the LLM $p_M(x_t \\mid c, x_{<t})$ and whose observation kernel is a Bernoulli emission $B(o_t; \\hat{r}(c, x_{\\le t}))$ scored by a process reward model. At each step the algorithm extends every particle with a sampled next step, assigns weight $w^{(i)}_t \\propto w^{(i)}_{t-1} \\hat{r}(c, x^{(i)}_{\\le t})$, and resamples the population by drawing indices from a softmax over these weights, which is what keeps exploration alive: low-scoring but potentially correct trajectories survive with nonzero probability. This weighted population is meant to approximate the posterior $\\hat{p}_M(x_{1:T} \\mid c, o_{1:T}=1)$ over accepted trajectories, and the paper uses that approximation to justify unbiased estimates of expected accuracy and to add MCMC-style extensions (Particle Gibbs, parallel tempering) that allocate compute across iterations and parallel chains.","core_discovery":"The paper's central claim is that re-framing inference-time scaling as posterior inference in a state-space model, with the LLM as the transition model and a process reward model as the approximate emission model, turns a brittle optimization problem into a well-posed sampling problem. The proposed particle filtering algorithm maintains N candidate trajectories, extends each by sampling from the LLM, scores partial trajectories with the PRM, and resamples through a softmax over those scores; the resulting population approximates the posterior over fully accepted sequences $\\hat{p}_M(x_{1:T} \\mid c, o_{1:T}=1)$, and sampling from this typical set instead of chasing the mode avoids the early-pruning failure of beam search. On MATH500 and AIME 2024 the authors find particle filtering consistently outperforms self-consistency, best-of-N, weighted best-of-N, beam search, and DVTS at equal compute budgets, and that its accuracy grows 4–16x faster with budget than the best search baseline. The headline results are that Qwen2.5-Math-1.5B-Instruct surpasses GPT-4o at a budget of 4 generations, and Qwen2.5-Math-7B-Instruct reaches 87.7% on MATH500 and 10/30 on AIME 2024 at a budget of 32, matching or exceeding o1-preview.","pith_inferences":["The paper leaves the softmax temperature in the resampling step fixed; treating it as a tunable exploration-exploitation knob, for example annealing it over the course of generation, is a natural extension that could improve scaling further on multi-modal tasks.","A corrected weight update that accumulates incremental likelihood products, $w_t \\propto w_{t-1} \\cdot \\hat{r}(x_{1:t})$, instead of raw PRM scores would align the algorithm with standard SMC and may make the unbiasedness proof go through cleanly; this is a testable modification of Algorithm 1.","If the early-pruning diagnosis is right, the advantage of particle filtering over beam search should grow as the reward model becomes noisier or as tasks have more genuinely distinct solution paths; comparing the two methods under controlled PRM noise levels would test that prediction.","The framework invites more advanced SMC tooling, such as twisted SMC with a learned value function or conditional particle filters, to reduce estimator variance."],"forward_implications":["On Llama-3.2-1B-Instruct, a budget of 8 particles matches what DVTS achieves with 32 generations, and on Llama-3.1-8B-Instruct the gap is 8 versus 128, so the same accuracy can be bought with 4–16x less test-time compute.","Small open math models become competitive with proprietary frontier models: Qwen2.5-Math-1.5B-Instruct surpasses GPT-4o on MATH500 with 4 rollouts, and Qwen2.5-Math-7B-Instruct reaches 87.7% on MATH500 and 10/30 on AIME 2024 with 32 rollouts, matching or exceeding o1-preview.","The probabilistic framing makes inference-time scaling a proper inference problem, so standard SMC theory becomes applicable, including unbiased estimators and MCMC kernels; the paper uses this to build Particle Gibbs and parallel-tempering variants that allocate compute across iterations and chains.","The method transfers beyond math: using a math-trained PRM as the reward model, particle filtering improves accuracy over all baselines on FinanceBench (70.33) and NumGLUE Task 2 Chemistry (84.22).","Because high-scoring candidates are favored but never allowed to dominate, the method avoids irreversible early pruning, meaning compute is not wasted on trajectories that the reward model initially underrates."],"supporting_citations":[{"why":"Supplies the dynamic variable-time search (DVTS) baseline and the budget-scaling evaluation setup the paper must beat.","marker":"[3]"},{"why":"Supplies best-of-N and weighted best-of-N baselines and the repeated-sampling framing for inference-time scaling.","marker":"[4]"},{"why":"Provides the sequential Monte Carlo theory that the particle filtering algorithm adapts and the basis for unbiasedness.","marker":"[7]"},{"why":"Provides the interacting particle resolution / particle filtering method that Algorithm 1 is built on.","marker":"[24]"},{"why":"Defines the state-space model formalism used to cast inference-time scaling as posterior inference.","marker":"[25]"},{"why":"Provides the Qwen2.5-Math-PRM-7B process reward model used to score partial trajectories in all main experiments.","marker":"[31]"},{"why":"Supplies the MATH500 benchmark used for the headline accuracy comparisons.","marker":"[16]"},{"why":"Supplies the beam search baseline and the compute-optimal test-time scaling perspective.","marker":"[23]"}],"fun_headline_variants":["Particle filtering beats beam search for LLM reasoning scaling","Particle filtering: 4-16x faster LLM reasoning scaling","Small model, big result: particle filtering hits GPT-4o in 4 rollouts","Particle filtering reaches o1-level with 32 rollouts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire probabilistic-interpretation claim rests on the softmax resampling step in Algorithm 1 being a valid sequential Monte Carlo weight update, so the particle population really converges to the posterior over accepted trajectories; the paper's Appendix C proof is a two-sentence assertion that never derives this from the algorithm's actual weight update.","fun_headline_variants_meta":{"raw":{"variants":["Particle filtering beats beam search for LLM reasoning scaling","Particle filtering: 4-16x faster LLM reasoning scaling","Small model, big result: particle filtering hits GPT-4o in 4 rollouts","Particle filtering reaches o1-level with 32 rollouts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001496,"raw_usage":{"total_tokens":6090,"prompt_tokens":1119,"completion_tokens":4971,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":735,"completion_tokens_details":{"reasoning_tokens":4893}},"tokens_in":735,"tokens_out":4971,"duration_ms":31207,"temperature":1.0,"reasoning_tokens":4893,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T14:47:53.769180+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Algorithm 1 on a small toy state-space model with enumerable states and compare the weighted-particle estimate of any expectation, such as expected accuracy, against the exact value computed from $\\hat{p}_M(x_{1:T} \\mid c, o_{1:T}=1)$; if the estimate stays biased as the number of particles grows, the unbiasedness theorem fails and the method is a stochastic search heuristic. A cheaper check is to inspect whether the implementation multiplies incremental likelihoods $w^{(i)}_t \\propto w^{(i)}_{t-1} \\hat{r}(c, x^{(i)}_{\\le t})$ or instead replaces the weight with the raw cumulative PRM score, since the pseudo-code in Algorithm 1 does the latter and that breaks the standard SMC recursion.","supporting_citations":[{"cited_title":"Le, Christopher Ré, and Azalia Mirhoseini","cited_arxiv_id":null,"evidence_quote":"Supplies best-of-N and weighted best-of-N baselines and the repeated-sampling framing for inference-time scaling."},{"cited_title":"Sequential Monte Carlo Methods for Dynamic Sys- tems: Journal of the American Statistical Association: V ol 93, No 443","cited_arxiv_id":null,"evidence_quote":"Provides the sequential Monte Carlo theory that the particle filtering algorithm adapts and the basis for unbiasedness."},{"cited_title":"Swendsen and Jian-Sheng Wang","cited_arxiv_id":null,"evidence_quote":"Provides the interacting particle resolution / particle filtering method that Algorithm 1 is built on."},{"cited_title":"Bayesian Filtering and Smoothing","cited_arxiv_id":null,"evidence_quote":"Defines the state-space model formalism used to cast inference-time scaling as posterior inference."}],"review_version":1}