{"id":"efcd6a75-9b2e-4503-ba28-4f8962ed654c","arxiv_id":"2505.17115","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"SIER uses kernel density estimation and Pareto-style selection to preserve diversity in PRM-guided multi-agent reasoning, improving pass@8 and prm@8 on math benchmarks at higher token cost.","lead":"This paper proposes a way to make groups of AI agents solve math problems better by borrowing a trick from swarm intelligence: explore diverse reasoning paths, not just good-sounding ones. The method scores each reasoning step with a reward model, keeps both high-quality and unusual steps, and reports gains on six math benchmarks plus a hard subset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported advantage over CoT and RGS is confounded by 4-8x higher token budgets in Table 1 and by comparing SIER pass@8/prm@8 with RGS sample@8; matched-compute, metric-matched baselines are needed before the central claim can be assessed.","rationale":"The most load-bearing requirement of the central claim is that SIER outperforms CoT and reward-guided search under a fixed evaluation budget. Table 1 violates this requirement in two ways: SIER uses 4-8x more tokens on the complex benchmarks where the gains appear, and the RGS baseline is scored with a different protocol that yields a single final answer. Therefore the empirical evidence does not currently isolate the density or non-dominated-sorting components. I partially agree with the reader's concern about the token-level KDE proxy; that is a real mechanism-level weakness, but it is downstream of the experimental confound. The paper's own ablation and unsolved-problem analyses are genuine supporting effort, but they are reported under the same unmatched evaluation protocol. The proposed matched-compute check is straightforward and would either strengthen or refute the headline claim. Because the paper could be revised to supply this evidence, I would keep the reader's conditional verdict rather than reject the work outright. No judgment about author intent is implied; the issue is that the table as reported does not control for the resources used.","tokens_in":19118,"tokens_out":11448,"duration_ms":98676,"concrete_test":"On AIME-2024 and MATH-500, rerun CoT with a matched token budget: since a CoT pass@8 sample costs about 1.08k tokens on AIME-2024, draw 38 independent CoT samples (about 40.8k tokens) and report pass@38 and prm@38; run RGS at the same token budget and use an 8-answer variant (or report SIER's single-final-answer accuracy) so the number of final answers is matched. If CoT pass@38/prm@38 or RGS at matched compute reaches or exceeds SIER's 26.7/23.3 on AIME-2024 and 93.0/86.2 on MATH-500, the claimed advantage is not attributable to the density-driven method.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 1 compares SIER's pass@8/prm@8 with CoT pass@8/prm@8 and RGS sample@8 at a fixed sample count k=8, but not at fixed compute. On AIME-2024, SIER spends 40.8k tokens versus 8.61k for CoT (4.7x) and 12.4k for RGS (3.3x); on MATH-500, SIER spends 32.3k versus 5.40k for CoT (6.0x) and 6.43k for RGS (5.0x); on the level-5 subset, 60.4k versus 7.04k (8.6x). The largest performance gaps occur on exactly the datasets where SIER consumes the most additional tokens, so the claim of consistent outperformance, especially on complex problems, can be explained by compute budget rather than by the density-diversity mechanism. In addition, the RGS baseline is metric-mismatched: RGS sample@8 returns one final answer per problem, while SIER pass@8 credits success if any of eight final answers is correct and SIER prm@8 selects among eight answers. This double confound prevents the headline experiments from isolating the contribution of the proposed method.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Agent-based Swarm Intelligence (ASI), a paradigm that recasts LLM reasoning as an optimization problem, and instantiated it as the SIER framework. SIER maintains a population of reasoning paths, constructs a token-level density landscape via kernel density estimation, selects next steps by non-dominated sorting over PRM-based quality and density, and clusters final solutions by extracted answer tags. Experiments on seven mathematical benchmarks report that SIER outperforms CoT and Reward-Guided Search (RGS) under pass@8, prm@8, and related metrics. The central claim is that density-guided diversity preservation, rather than raw sampling or step-level greedy reward search, drives the gains, particularly on difficult problems.","tokens_in":19370,"tokens_out":3282,"duration_ms":30308,"significance":"If the central claim is upheld, the paper would contribute a useful conceptual bridge between evolutionary multimodal optimization and LLM reasoning, with a concrete mechanism (density-assisted Pareto step selection) that is easy to state and potentially portable to other inference-time search schemes. The supplementary analysis of MAD diversity (Appendix B) is a genuine strength, as it gives a measurable motivation for the diversity objective. The ablations in Table 2 go beyond a single headline number and indicate that the proposed components jointly contribute. However, the paper does not currently support its headline 'consistently outperforms' claim: the main comparison is confounded by token budget and by mismatched evaluation metrics, key hyperparameters are undisclosed, and the prm@8 metric shares its reward model with the search process. Because these issues bear directly on the central empirical claim, the paper needs substantial revision before the claimed result can be accepted.","major_comments":[{"comment":"The headline comparisons are confounded by compute budget. SIER consumes 40.8k tokens on AIME-2024 versus 12.4k for RGS and 8.61k for CoT, and 60.4k versus 10.5k and 7.04k on MATH-500 (level-5); the largest performance gaps occur exactly on the benchmarks with the largest token gaps. The paper acknowledges the extra tokens but treats them as inherent to the method rather than as a controlled variable. To support the claim of consistent outperformance, the authors should add matched-compute baselines (e.g., CoT and RGS with comparable token budgets, or SIER with a capped budget) and report both pass@8 and prm@8 for all methods at each budget. Without such a comparison, the observed gains can be explained by additional sampling rather than by the density-diversity mechanism.","section":"Table 1, Section 5.2"},{"comment":"The RGS baseline is metric-mismatched with SIER. RGS is evaluated with sample@8, which selects one step per position and returns a single final answer, whereas SIER is evaluated with pass@8 and prm@8, which count a problem as solved if any of eight final answers is correct or if the best of eight answers is selected. This double confound makes the comparison unfair in both directions and prevents any statement that SIER outperforms RGS. The authors should report a matched protocol, for example SIER under a sample@8-style protocol (single path per problem) or RGS under pass@8/prm@8 on the same eight sampled trajectories, and ideally both.","section":"Section 5.1, Table 1"},{"comment":"Several load-bearing hyperparameters are either unspecified or tuned on the evaluation benchmarks themselves. The KDE bandwidth h and the small batch size b_s appear in Algorithms 2 and 3 but their values are never given, so the reported results are not reproducible. Additionally, the quality threshold theta is set to 0.99 by default but Appendix F reports tuning over 0.5-0.99 using the same AIME and MATH datasets, with no separate validation split. This makes the headline results vulnerable to selection on the test data. Please disclose h and b_s, and either tune theta on a held-out set or present results across theta values without treating the best theta as the method's default.","section":"Section 5.1, Appendix F"},{"comment":"The prm@8 metric is partially circular with respect to the proposed mechanism. The same PRM (Qwen2.5-Math-PRM-72B) is used both to steer step selection inside SIER and to select the best final answer for prm@8, so prm@8 measures alignment with the optimizer's preferences rather than independent correctness. The paper would be substantially stronger if it reported, in addition to prm@8, an independent selection signal (e.g., outcome-reward-model selection or majority voting) and verified that the density-driven search still improves over CoT and RGS under that signal. The limitation section correctly notes that PRM bias can misdirect evolution, but it does not address this circularity in the evaluation metric.","section":"Section 5.1, Section 7"},{"comment":"The experimental results are reported as single numbers without error bars, repeated seeds, or significance tests, yet many of the claimed wins are small (e.g., MMLU-STEM pass@8 92.8 vs. 91.7, GSM8K pass@8 97.4 vs. 97.0). Given the large variance typical of eight-sample CoT estimates on these benchmarks, one cannot determine whether the differences are systematic or noise. Please report means and standard deviations over multiple runs (at least three seeds) and, where feasible, a paired test over problems, at least for the main Table 1 comparisons.","section":"Tables 1 and 2"}],"minor_comments":[{"comment":"There are typos in the text: 'we propos a novel paradigm' and 'conceptualized the LLM's reasoning process' should be 'propose' and 'conceptualizes' respectively; Algorithm 2's header also spells 'Density Calcualation' instead of 'Calculation'.","section":"Section 4"},{"comment":"The pseudocode contains undefined or inconsistent variables: in the inner loop, candidate nodes are indexed as n_cand_j but the early-stop branch refers to n_j, e_m, and e_p, and line 31 sets e_mtotal to e_p,|u| with no corresponding update of the individual record. Please revise Algorithm 3 so that all variables are defined and the early-exit and history-update paths match the prose description.","section":"Algorithm 3"},{"comment":"The KDE operates on token identities and step indices, but the solution space is described as having each dimension correspond to a token with values from the vocabulary. This makes the distance metric and the notion of 'low-density region' hard to interpret. Please specify the exact distance used in the truncated Gaussian kernel, how token history is aggregated (e.g., whether repeated tokens at different step indices are treated as separate observations), and why token-level density is expected to proxy semantic diversity rather than lexical frequency.","section":"Section 3.1, Appendix D.2.1"},{"comment":"In Table 4, the note says 'CoT and SIER are evaluated with sample@8', while Section 5.1 defines sample@k as step-level selection and the main table reports SIER under pass@8 and prm@8. This notation is inconsistent and may confuse readers; please align the terminology or clarify that the appendix uses a different protocol.","section":"Table 4 and Section 5.1"},{"comment":"Figure 4 reports performance on 'unsolved problems' without stating the number of problems in each subset, so the reader cannot judge the reliability of percentages such as 15.4% versus 7.69% on AIME-2024. Please report the denominator or add error bars.","section":"Section 5.4, Figure 4"},{"comment":"The sentence 'Specifically, the maximum number of iterations in the evolutionary phase is set to 1. This means that the task has been solved' is confusing; it should say that the evolution phase is skipped when the highest quality of the initial population exceeds theta, while the maximum iteration count is relevant only for problems that do not meet the threshold.","section":"Section 5.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a preprint under review and no code or data are released, which compounds the reproducibility problems noted in the major comments. The novelty lies mainly in transferring density-based niching ideas from evolutionary multimodal optimization to LLM reasoning; the empirical validation needs to be substantially reworked with matched-compute, metric-matched, and tuned-on-holdout comparisons before the 'consistently outperforms' claim can be taken at face value."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on SIER. The core idea is a real combination: treat LLM reasoning as a swarm search, use KDE on token sequences to build a density landscape, and use non-dominated sorting to pick steps that balance PRM quality and low density. That specific integration is new and the paper does a decent job explaining it. The ablations support each component, and the unsolved-problems analysis is a nice touch.\n\nThe soft spot is the evaluation. Table 1 compares SIER's pass@8/prm@8 against CoT's pass@8/prm@8 and RGS's sample@8, all at k=8, but SIER burns 4-8x more tokens. On AIME-2024 it's 40.8k vs 8.61k for CoT and 12.4k for RGS; on MATH-500 level-5 it's 60.4k vs 7.04k. The biggest gains land exactly where the token ratio is largest, so the 'consistent outperformance' might mostly be more compute. The RGS comparison is also metric-mismatched: sample@8 produces one final answer per problem while pass@8 counts success if any of eight answers is right. That double confound is the main reason I can't take the headline numbers at face value.\n\nSmaller issues: no error bars, theta is tuned on the same benchmarks it's evaluated on, and hyperparameters like KDE bandwidth h and batch size b_s aren't reported. The density proxy—token-level KDE approximating semantic diversity—is plausible but unvalidated; an analysis showing that chosen low-density steps actually lead to diverse correct solutions would help. The authors do acknowledge token cost and PRM bias, which is honest, but the paragraph in Section 5.2 treating extra tokens as inherent doesn't resolve the confound.\n\nWho's this for: people working on inference-time search for LLMs, process reward models, and multi-agent reasoning. It deserves a serious referee. The right move is to send it to review with a request for matched-compute baselines (e.g., CoT with 8 or 40 samples, RGS at matched token budget), metric-matched RGS comparison, multiple seeds, and released code with concrete hyperparameters. If those hold up, the result is solid; if not, the contribution shrinks to a modest diversity heuristic. Either way it's worth referee time.","headline":"Genuine but compute-confounded: SIER's density-driven step selection needs matched-budget baselines before the central claim is credible.","tokens_in":19932,"tokens_out":2296,"would_cite":false,"duration_ms":19123,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T50","90C29","62G07"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that treating LLM reasoning as a swarm-intelligence search over reasoning paths, with kernel-density and Pareto-style step selection, lets agent populations consistently beat chain-of-thought and reward-guided baselines…","keywords":["swarm intelligence","large language models","multi-agent systems","mathematical reasoning","kernel density estimation","non-dominated sorting","process reward model","chain-of-thought"],"falsifier":"On a held-out set of difficult problems, record the density of every candidate step at selection time and compare whether low-density steps lead to correct final answers more often than high-density steps. If low-density steps are no more productive than high-density ones, or if a selection rule based on semantic diversity matches SIER's pass@8 and prm@8 without the density term, then the density mechanism is not what produces the reported gains.","tokens_in":18892,"feed_emoji":"🐝","tokens_out":10976,"duration_ms":89994,"temperature":0.7,"pith_summary":"The paper is trying to establish that a population of LLM agents can solve hard reasoning problems by treating reasoning as an optimization search rather than as a single chain of thought. Its SIER framework has agents expand many reasoning paths in parallel, scores each step with a process reward model, and builds a density map of already-explored tokens so that the swarm preferentially expands into under-explored directions. Candidate next steps are then chosen by non-dominated sorting over quality and density, which keeps both accuracy and diversity in the population instead of collapsing onto one path. The authors report that this consistently beats plain chain-of-thought sampling and step-level reward-guided search on seven mathematical benchmarks, with the largest gains on the hardest problems such as AIME and the level-5 subset of MATH-500. If the claim holds, diversity becomes a controllable search resource rather than a random by-product of sampling.","feed_headline":"Density-guided agent swarm beats chain-of-thought on math tests","feed_subtitle":"Multi-agent search balancing answer quality and diversity beats CoT and reward-guided baselines on hard reasoning.","key_machinery":"The load-bearing mechanism is the density landscape constructed with truncated Gaussian kernel density estimation over token sequences. Each reasoning path is split into tokens, each token is assigned the step index at which it occurs, and the density of a token at the current step is the sum of truncated Gaussian contributions from all historical occurrences, scaled by the ratio of the step-index difference to the bandwidth; fully explored tokens get high density and under-explored tokens get low density. Fast non-dominated sorting then ranks candidate steps by the two objectives of PRM step quality and negative density, and only the first non-dominated front is expanded. This converts 'keep solutions diverse' from a heuristic into a concrete selection rule, while the quality threshold $\\theta$ makes the search stop or resample based on how confident the evaluator is.","core_discovery":"SIER's central discovery is that the diversity deficit in multi-agent LLM reasoning can be formulated as a multimodal optimization problem and attacked with density-assisted selection. The framework starts a population of independent reasoning paths, evaluates each intermediate step with a PRM, and constructs a kernel-density landscape in which every token carries a density value based on how often and at which step it has appeared in the historical population. At each expansion, k candidate steps are sampled; their PRM quality scores and their densities are treated as two objectives, and fast non-dominated sorting keeps only the Pareto-optimal steps, so a step survives only if no alternative is both higher-quality and lower-density. Quality thresholds then stop the evolution early when a very good solution exists and trigger additional resampling when sampled steps are weak. On the paper's seven benchmarks, this selection rule yields higher pass@8 and prm@8 than both CoT and RGS, and the ablation results attribute the gains jointly to fitness, density, and the evolutionary mechanism.","pith_inferences":["A direct test of the core assumption would be to measure, on held-out problems, whether low-density candidate steps are more likely than high-density ones to lead to correct answers; if not, a semantic-diversity selector might match SIER at lower cost.","Because the method is not math-specific, the same density-assisted swarm search could be applied to code generation, planning, or any stepwise task that has a step-level reward signal.","The paper's own analysis shows the PRM is the weak point on LiveMathBench (87.5% identification success); using an ensemble of PRMs in the density-guided selection would directly test whether evaluator bias, rather than search, causes the remaining errors.","Token-step density ignores semantic similarity, so two tokens that mean nearly the same thing may be counted as far apart; replacing the token-step metric with embedding-space density would reveal whether the reported gains come from diversity itself or from the specific density definition."],"forward_implications":["On the paper's benchmarks, SIER raises both the chance that at least one sampled solution is correct (pass@8) and the chance that a PRM picks a correct solution (prm@8), relative to CoT and reward-guided search.","The diversity mechanism matters most where problems are hard: the largest margins over CoT appear on AIME-2024, AIME-2025, and MATH-500 level-5, where single-path or greedy methods stagnate.","A high quality threshold (near 0.99) is the effective operating point, because lower thresholds rarely trigger the evolutionary resampling that drives the gains.","The framework uses the same plain step-by-step prompt as CoT and gains from search structure rather than role-play prompts, so the approach is model-agnostic as long as the base model supports step-by-step continuation.","Token cost scales with problem difficulty, reaching roughly five times the CoT cost on MATH-500; the paper presents this as the price of broader exploration rather than a flaw."],"supporting_citations":[{"why":"Supplies the process reward model, the RGS baseline, and the sampling configuration that SIER builds on and is compared against.","marker":"[46]"},{"why":"Supplies the truncated Gaussian kernel density estimation method used to build the density landscape over tokens.","marker":"[42]"},{"why":"Supplies the fast non-dominated sorting procedure used to rank candidate steps by quality and density.","marker":"[8]"},{"why":"Provides the evidence that multi-agent debate methods often underperform CoT, motivating the diversity-first design.","marker":"[45]"},{"why":"Supplies the self-consistency majority-voting baseline (major@k) against which SIER's answer selection is measured.","marker":"[37]"},{"why":"Supplies the GSM8K benchmark used to evaluate mathematical reasoning performance.","marker":"[7]"},{"why":"Supplies the MATH dataset from which the MATH-500 and level-5 subsets are drawn.","marker":"[15]"},{"why":"Supplies the AIME-2024 benchmark used as a hard reasoning test set.","marker":"[27]"},{"why":"Supplies the AIME-2025 benchmark used as a hard reasoning test set.","marker":"[28]"}],"fun_headline_variants":["Density-guided LLM swarm beats chain-of-thought on math benchmarks","Swarm optimizer for LLMs: density-aware Pareto steps improve reasoning","Balancing diversity and quality: LLM swarm reasoning outperforms CoT","Density-driven multi-agent framework outperforms CoT on seven math tasks","LLM reasoning as optimization: swarm plus density beats baselines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the token-density landscape computed by kernel density estimation accurately marks which reasoning directions are genuinely under-explored, so steering the swarm toward low-density steps actually uncovers new correct answers rather than merely unusual wording.","fun_headline_variants_meta":{"raw":{"variants":["Density-guided LLM swarm beats chain-of-thought on math benchmarks","Swarm optimizer for LLMs: density-aware Pareto steps improve reasoning","Balancing diversity and quality: LLM swarm reasoning outperforms CoT","Density-driven multi-agent framework outperforms CoT on seven math tasks","LLM reasoning as optimization: swarm plus density beats baselines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000489,"raw_usage":{"total_tokens":2438,"prompt_tokens":1007,"completion_tokens":1431,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":623,"completion_tokens_details":{"reasoning_tokens":1338}},"tokens_in":623,"tokens_out":1431,"duration_ms":9921,"temperature":1.0,"reasoning_tokens":1338,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:13:01.593003+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a held-out set of difficult problems, record the density of every candidate step at selection time and compare whether low-density steps lead to correct final answers more often than high-density steps. If low-density steps are no more productive than high-density ones, or if a selection rule based on semantic diversity matches SIER's pass@8 and prm@8 without the density term, then the density mechanism is not what produces the reported gains.","supporting_citations":[{"cited_title":"An alternative way of evolutionary multimodal optimization: density-based population initialization strategy","cited_arxiv_id":null,"evidence_quote":"Supplies the truncated Gaussian kernel density estimation method used to build the density landscape over tokens."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the fast non-dominated sorting procedure used to rank candidate steps by quality and density."},{"cited_title":"American invitational mathematics examination - aime","cited_arxiv_id":null,"evidence_quote":"Supplies the AIME-2024 benchmark used as a hard reasoning test set."},{"cited_title":"American invitational mathematics examination - aime","cited_arxiv_id":null,"evidence_quote":"Supplies the AIME-2025 benchmark used as a hard reasoning test set."}],"review_version":1}