Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Swarm Intelligence Enhanced Reasoning: A Density-Driven Framework for LLM-Based Multi-Agent Optimization

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that treating LLM reasoning as a swarm-intelligence search over reasoning paths, with kernel-density and Pareto-style step selection, lets agent populations consistently beat chain-of-thought and reward-guided baselines…

desk verdict Genuine but compute-confounded: SIER's density-driven step selection needs matched-budget baselines before the central claim is credible. read the letter →

arxiv 2505.17115 v2 pith:Q2AKUTI7 submitted 2025-05-21 cs.MA cs.AI

Ying Zhu , Heng Zhou , Rui Su , Peiqin Zhuang , Lei Bai This is my paper · ORCID
classification cs.MAcs.AI MSC 68T5090C2962G07
keywords swarmintelligencelargelanguagemodelsmulti-agentsystemsmathematicalreasoningkerneldensityestimationnon-dominatedsortingprocessrewardmodelchain-of-thought
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a population of LLM agents can solve hard reasoning problems by treating reasoning as an optimization search rather than as a single chain of thought. Its SIER framework has agents expand many reasoning paths in parallel, scores each step with a process reward model, and builds a density map of already-explored tokens so that the swarm preferentially expands into under-explored directions. Candidate next steps are then chosen by non-dominated sorting over quality and density, which keeps both accuracy and diversity in the population instead of collapsing onto one path. The authors report that this consistently beats plain chain-of-thought sampling and step-level reward-guided search on seven mathematical benchmarks, with the largest gains on the hardest problems such as AIME and the level-5 subset of MATH-500. If the claim holds, diversity becomes a controllable search resource rather than a random by-product of sampling.

What carries the argument

The load-bearing mechanism is the density landscape constructed with truncated Gaussian kernel density estimation over token sequences. Each reasoning path is split into tokens, each token is assigned the step index at which it occurs, and the density of a token at the current step is the sum of truncated Gaussian contributions from all historical occurrences, scaled by the ratio of the step-index difference to the bandwidth; fully explored tokens get high density and under-explored tokens get low density. Fast non-dominated sorting then ranks candidate steps by the two objectives of PRM step quality and negative density, and only the first non-dominated front is expanded. This converts 'keep solutions diverse' from a heuristic into a concrete selection rule, while the quality threshold $\theta$ makes the search stop or resample based on how confident the evaluator is.

What would settle it

On a held-out set of difficult problems, record the density of every candidate step at selection time and compare whether low-density steps lead to correct final answers more often than high-density steps. If low-density steps are no more productive than high-density ones, or if a selection rule based on semantic diversity matches SIER's pass@8 and prm@8 without the density term, then the density mechanism is not what produces the reported gains.

Watch

Extended reading notes

Core claim

SIER's central discovery is that the diversity deficit in multi-agent LLM reasoning can be formulated as a multimodal optimization problem and attacked with density-assisted selection. The framework starts a population of independent reasoning paths, evaluates each intermediate step with a PRM, and constructs a kernel-density landscape in which every token carries a density value based on how often and at which step it has appeared in the historical population. At each expansion, k candidate steps are sampled; their PRM quality scores and their densities are treated as two objectives, and fast non-dominated sorting keeps only the Pareto-optimal steps, so a step survives only if no alternative is both higher-quality and lower-density. Quality thresholds then stop the evolution early when a very good solution exists and trigger additional resampling when sampled steps are weak. On the paper's seven benchmarks, this selection rule yields higher pass@8 and prm@8 than both CoT and RGS, and the ablation results attribute the gains jointly to fitness, density, and the evolutionary mechanism.

Load-bearing premise

The load-bearing premise is that the token-density landscape computed by kernel density estimation accurately marks which reasoning directions are genuinely under-explored, so steering the swarm toward low-density steps actually uncovers new correct answers rather than merely unusual wording.

Editorial extensions

If this is right

  • On the paper's benchmarks, SIER raises both the chance that at least one sampled solution is correct (pass@8) and the chance that a PRM picks a correct solution (prm@8), relative to CoT and reward-guided search.
  • The diversity mechanism matters most where problems are hard: the largest margins over CoT appear on AIME-2024, AIME-2025, and MATH-500 level-5, where single-path or greedy methods stagnate.
  • A high quality threshold (near 0.99) is the effective operating point, because lower thresholds rarely trigger the evolutionary resampling that drives the gains.
  • The framework uses the same plain step-by-step prompt as CoT and gains from search structure rather than role-play prompts, so the approach is model-agnostic as long as the base model supports step-by-step continuation.
  • Token cost scales with problem difficulty, reaching roughly five times the CoT cost on MATH-500; the paper presents this as the price of broader exploration rather than a flaw.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the core assumption would be to measure, on held-out problems, whether low-density candidate steps are more likely than high-density ones to lead to correct answers; if not, a semantic-diversity selector might match SIER at lower cost.
  • Because the method is not math-specific, the same density-assisted swarm search could be applied to code generation, planning, or any stepwise task that has a step-level reward signal.
  • The paper's own analysis shows the PRM is the weak point on LiveMathBench (87.5% identification success); using an ensemble of PRMs in the density-guided selection would directly test whether evaluator bias, rather than search, causes the remaining errors.
  • Token-step density ignores semantic similarity, so two tokens that mean nearly the same thing may be counted as far apart; replacing the token-step metric with embedding-space density would reveal whether the reported gains come from diversity itself or from the specific density definition.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The manuscript proposes Agent-based Swarm Intelligence (ASI), a paradigm that recasts LLM reasoning as an optimization problem, and instantiated it as the SIER framework. SIER maintains a population of reasoning paths, constructs a token-level density landscape via kernel density estimation, selects next steps by non-dominated sorting over PRM-based quality and density, and clusters final solutions by extracted answer tags. Experiments on seven mathematical benchmarks report that SIER outperforms CoT and Reward-Guided Search (RGS) under pass@8, prm@8, and related metrics. The central claim is that density-guided diversity preservation, rather than raw sampling or step-level greedy reward search, drives the gains, particularly on difficult problems.

Significance. If the central claim is upheld, the paper would contribute a useful conceptual bridge between evolutionary multimodal optimization and LLM reasoning, with a concrete mechanism (density-assisted Pareto step selection) that is easy to state and potentially portable to other inference-time search schemes. The supplementary analysis of MAD diversity (Appendix B) is a genuine strength, as it gives a measurable motivation for the diversity objective. The ablations in Table 2 go beyond a single headline number and indicate that the proposed components jointly contribute. However, the paper does not currently support its headline 'consistently outperforms' claim: the main comparison is confounded by token budget and by mismatched evaluation metrics, key hyperparameters are undisclosed, and the prm@8 metric shares its reward model with the search process. Because these issues bear directly on the central empirical claim, the paper needs substantial revision before the claimed result can be accepted.

major comments (5)
  1. [Table 1, Section 5.2] The headline comparisons are confounded by compute budget. SIER consumes 40.8k tokens on AIME-2024 versus 12.4k for RGS and 8.61k for CoT, and 60.4k versus 10.5k and 7.04k on MATH-500 (level-5); the largest performance gaps occur exactly on the benchmarks with the largest token gaps. The paper acknowledges the extra tokens but treats them as inherent to the method rather than as a controlled variable. To support the claim of consistent outperformance, the authors should add matched-compute baselines (e.g., CoT and RGS with comparable token budgets, or SIER with a capped budget) and report both pass@8 and prm@8 for all methods at each budget. Without such a comparison, the observed gains can be explained by additional sampling rather than by the density-diversity mechanism.
  2. [Section 5.1, Table 1] The RGS baseline is metric-mismatched with SIER. RGS is evaluated with sample@8, which selects one step per position and returns a single final answer, whereas SIER is evaluated with pass@8 and prm@8, which count a problem as solved if any of eight final answers is correct or if the best of eight answers is selected. This double confound makes the comparison unfair in both directions and prevents any statement that SIER outperforms RGS. The authors should report a matched protocol, for example SIER under a sample@8-style protocol (single path per problem) or RGS under pass@8/prm@8 on the same eight sampled trajectories, and ideally both.
  3. [Section 5.1, Appendix F] Several load-bearing hyperparameters are either unspecified or tuned on the evaluation benchmarks themselves. The KDE bandwidth h and the small batch size b_s appear in Algorithms 2 and 3 but their values are never given, so the reported results are not reproducible. Additionally, the quality threshold theta is set to 0.99 by default but Appendix F reports tuning over 0.5-0.99 using the same AIME and MATH datasets, with no separate validation split. This makes the headline results vulnerable to selection on the test data. Please disclose h and b_s, and either tune theta on a held-out set or present results across theta values without treating the best theta as the method's default.
  4. [Section 5.1, Section 7] The prm@8 metric is partially circular with respect to the proposed mechanism. The same PRM (Qwen2.5-Math-PRM-72B) is used both to steer step selection inside SIER and to select the best final answer for prm@8, so prm@8 measures alignment with the optimizer's preferences rather than independent correctness. The paper would be substantially stronger if it reported, in addition to prm@8, an independent selection signal (e.g., outcome-reward-model selection or majority voting) and verified that the density-driven search still improves over CoT and RGS under that signal. The limitation section correctly notes that PRM bias can misdirect evolution, but it does not address this circularity in the evaluation metric.
  5. [Tables 1 and 2] The experimental results are reported as single numbers without error bars, repeated seeds, or significance tests, yet many of the claimed wins are small (e.g., MMLU-STEM pass@8 92.8 vs. 91.7, GSM8K pass@8 97.4 vs. 97.0). Given the large variance typical of eight-sample CoT estimates on these benchmarks, one cannot determine whether the differences are systematic or noise. Please report means and standard deviations over multiple runs (at least three seeds) and, where feasible, a paired test over problems, at least for the main Table 1 comparisons.
minor comments (6)
  1. [Section 4] There are typos in the text: 'we propos a novel paradigm' and 'conceptualized the LLM's reasoning process' should be 'propose' and 'conceptualizes' respectively; Algorithm 2's header also spells 'Density Calcualation' instead of 'Calculation'.
  2. [Algorithm 3] The pseudocode contains undefined or inconsistent variables: in the inner loop, candidate nodes are indexed as n_cand_j but the early-stop branch refers to n_j, e_m, and e_p, and line 31 sets e_mtotal to e_p,|u| with no corresponding update of the individual record. Please revise Algorithm 3 so that all variables are defined and the early-exit and history-update paths match the prose description.
  3. [Section 3.1, Appendix D.2.1] The KDE operates on token identities and step indices, but the solution space is described as having each dimension correspond to a token with values from the vocabulary. This makes the distance metric and the notion of 'low-density region' hard to interpret. Please specify the exact distance used in the truncated Gaussian kernel, how token history is aggregated (e.g., whether repeated tokens at different step indices are treated as separate observations), and why token-level density is expected to proxy semantic diversity rather than lexical frequency.
  4. [Table 4 and Section 5.1] In Table 4, the note says 'CoT and SIER are evaluated with sample@8', while Section 5.1 defines sample@k as step-level selection and the main table reports SIER under pass@8 and prm@8. This notation is inconsistent and may confuse readers; please align the terminology or clarify that the appendix uses a different protocol.
  5. [Section 5.4, Figure 4] Figure 4 reports performance on 'unsolved problems' without stating the number of problems in each subset, so the reader cannot judge the reliability of percentages such as 15.4% versus 7.69% on AIME-2024. Please report the denominator or add error bars.
  6. [Section 5.4] The sentence 'Specifically, the maximum number of iterations in the evolutionary phase is set to 1. This means that the task has been solved' is confusing; it should say that the evolution phase is skipped when the highest quality of the initial population exceeds theta, while the maximum iteration count is relevant only for problems that do not meet the threshold.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: pass@8 is an independent exact-answer metric and the framework is specified in the paper; the main caveats are comparison-design confounds, not definitional circularity.

full rationale

The paper's central claim is an empirical comparison, not a derivation from first principles, and the headline pass@8 metric is computed against exact boxed answers, so it does not reduce to the PRM that guides SIER. No fitted parameter is extracted from the target benchmarks and then renamed a prediction: the main algorithmic choices (k=8, theta=0.99, Imax=1, KDE bandwidth h) are fixed settings, and the ablations in Table 2 vary whole mechanisms rather than fitting to outcomes. The prm@8 metric does reuse the same PRM used for step selection and final ranking, so it partly measures alignment with the optimizer rather than independent quality; however, the selected answer is still checked against ground truth, and this is a standard reward-model evaluation design, not a self-definitional reduction. The token-budget gap between SIER and CoT/RGS (e.g., MATH-500 level-5: 60.4k vs 7.04k tokens in Table 1) and the pass@8-versus-sample@8 metric mismatch are real threats to the comparative claim, but they are validity and confounding concerns, not circularity. Reference [50] is a self-citation by the first author, but it appears only in related-work discussions of diversity-maintenance strategies; the density landscape and Pareto selection are fully specified in Algorithms 2-4 using standard KDE and non-dominated sorting, and their contribution is independently ablated in Table 2. No uniqueness theorem is imported, no ansatz is justified solely by self-citation, and no known result is merely renamed. The manuscript's own limitation section explicitly concedes PRM bias and search-efficiency issues, which further supports that the claimed mechanism is treated as empirically testable rather than true by construction.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The central claim depends on threshold theta, sample count, iteration count, KDE bandwidth, batch size, and the unvalidated token-density proxy. Most are not derived and some are not even specified, so the upstream burden is substantial.

free parameters (6)
  • quality threshold theta = 0.99
    Controls whether the evolution phase is skipped and whether a sampled step is accepted immediately. Tuned by sweeping 0.5 to 0.99 on the same benchmarks (Section 5.5, Appendix F.1).
  • sample count k = 8
    Number of candidate steps sampled per step and number of final solutions; set in Section 5.1, with no sensitivity analysis.
  • maximum evolution iterations Imax = 1
    Number of population evolution rounds; fixed to 1 in Section 5.4 and used in Algorithm 3.
  • KDE bandwidth h
    Bandwidth of the truncated Gaussian kernel in Algorithm 2; critical for density values but no numeric value is given.
  • small batch size b_s
    Used in Algorithm 3 to check the early-acceptance threshold; no value is provided.
  • step delimiter = \n\n
    Stop word used to segment reasoning steps into generations (Appendix E.2); chosen by the authors.
assumptions (3)
  • domain assumption Token-level KDE density over historical solutions approximates the diversity of reasoning paths in a useful way.
    Section 4.3 and Algorithm 2 build the density landscape from token and step indices; the claimed diversity benefit depends on this proxy being semantically meaningful.
  • domain assumption Process reward model scores are reliable enough to guide population evolution toward correct solutions.
    The evaluator E (Qwen2.5-Math-PRM-72B) supplies the quality objective in non-dominated sorting; the authors themselves flag PRM bias as a limitation in Section 7 and Appendix G.
  • domain assumption LLM reasoning can be modeled as search over a solution space where each dimension is a token and solutions are reasoning paths.
    Section 4 introduces this formulation; no metric or topological structure is provided, so it is a modeling assumption rather than a proven fact.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Swarm Intelligence Enhanced Reasoning: A Density-Driven Framework for LLM-Based Multi-Agent Optimization." pith.science (2026). https://pith.science/paper/Q2AKUTI7

@misc{pith2026250517115,
  author       = {Pith},
  title        = {Pith review of: Swarm Intelligence Enhanced Reasoning: A Density-Driven Framework for LLM-Based Multi-Agent Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q2AKUTI7}},
  note         = {Machine review of arXiv:2505.17115}
}
read the original abstract

Recently, many approaches, such as Chain-of-Thought (CoT) prompting and Multi-Agent Debate (MAD), have been proposed to further enrich Large Language Models' (LLMs) complex problem-solving capacities in reasoning scenarios. However, these methods may fail to solve complex problems due to the lack of ability to find optimal solutions. Swarm Intelligence has been serving as a powerful tool for finding optima in the field of traditional optimization problems. To this end, we propose integrating swarm intelligence into the reasoning process by introducing a novel Agent-based Swarm Intelligence (ASI) paradigm. In this paradigm, we formulate LLM reasoning as an optimization problem and use a swarm intelligence scheme to guide a group of LLM-based agents in collaboratively searching for optimal solutions. To avoid swarm intelligence getting trapped in local optima, we further develop a Swarm Intelligence Enhancing Reasoning (SIER) framework, which develops a density-driven strategy to enhance the reasoning ability. To be specific, we propose to perform kernel density estimation and non-dominated sorting to optimize both solution quality and diversity simultaneously. In this case, SIER efficiently enhances solution space exploration through expanding the diversity of the reasoning path. Besides, a step-level quality evaluation is used to help agents improve solution quality by correcting low-quality intermediate steps. Then, we use quality thresholds to dynamically control the termination of exploration and the selection of candidate steps, enabling a more flexible and efficient reasoning process. Extensive experiments are ...

Figures

Figures reproduced from arXiv: 2505.17115 by the authors.

Figure 1
Figure 1. Visualization of Pareto-optimal solutions and non-dominated sorting [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Overview of the SIER framework. 4.2 Population Initialization Given a task query q, our SIER framework first generates the initial population consisting of n individuals. Specifically, n LLM-based agents are created as individuals. Each agent independently processes the input q to generate a distinct step-by-step reasoning path. These reasoning paths collectively form the population’s initial solution set. Then, a s… view at source ↗
Figure 3
Figure 3. The percentage of unsolved problems identified by [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Performance comparison of SIER, RGS, and CoT variants on the unsolved problems. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Impact of Different Quality Threshold θ on SIER’s Performance (Log Scale). F.2 Tree Search Strategy - Reasoning via Planning Reasoning via Planning (RAP) is a new reasoning framework for Large Language Models (LLMs) that aims to overcome the shortcomings of LLMs in gen…
Figure 6
Figure 6. Figure 6: Performance comparison of SIER and RAP. According to the experimental results, the SIER method proposed in this paper outperforms the RAP method on most datasets, especially on the GSM8K and LiveMathBench datasets. Specifically, the RAP method is inferior to SIER in te…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Swarm-Driven Multi-Agent Reasoning for Smart City Security

    cs.CR 2026-07 conditional novelty 5.0 of 10

    Specialized LLM agents plus Threat-Pheromone Swarm Consensus (and AV-TPSC) yield high consensus stability and better Acc/F1 than single-agent or majority-vote baselines on ToN-IoT-style smart-city evidence.

Reference graph

Works this paper leans on

50 extracted references · 29 canonical work pages · cited by 1 Pith paper

  1. [1]

    Ali Ahrari, Saber Elsayed, Ruhul Sarker, Daryl Essam, and Carlos A Coello Coello. Static and dynamic multimodal optimization by improved covariance matrix self-adaptation evolution strategy with repelling subpopulations.IEEE Transactions on Evolutionary Computation, 26(3):527–541, 2022

    work page 2022

  2. [2]

    Number 1

    Eric Bonabeau, Marco Dorigo, and Guy Theraulaz.Swarm intelligence: from natural to artificial systems. Number 1. Oxford university press, 1999

    work page 1999

  3. [3]

    Chateval: Towards better llm-based evaluators through multi-agent debate

    Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better llm-based evaluators through multi-agent debate. In The Twelfth International Conference on Learning Representations

    work page

  4. [4]

    Comm: Collaborative multi-agent, multi-reasoning- path prompting for complex problem solving

    Pei Chen, Shuai Zhang, and Boran Han. Comm: Collaborative multi-agent, multi-reasoning- path prompting for complex problem solving. InFindings of the Association for Computational Linguistics: NAACL 2024, pages 1720–1738, 2024

    work page 2024

  5. [5]

    Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors

    Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, et al. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. InThe Twelfth International Conference on Learning Representations

    work page

  6. [6]

    Ran Cheng, Miqing Li, Ke Li, and Xin Yao. Evolutionary multiobjective optimization-based multimodal optimization: Fitness landscape approximation and peak detection.IEEE Transac- tions on Evolutionary Computation, 22(5):692–706, 2018

    work page 2018

  7. [7]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    arXiv

  8. [8]

    K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: Nsga-ii.IEEE Transactions on Evolutionary Computation, 6(2):182–197, 2002

    work page 2002

Show all 50 references
  1. [9]

    Ant colony optimization.IEEE computa- tional intelligence magazine, 1(4):28–39, 2007

    Marco Dorigo, Mauro Birattari, and Thomas Stutzle. Ant colony optimization.IEEE computa- tional intelligence magazine, 1(4):28–39, 2007

    2007

  2. [10]

    Improv- ing factuality and reasoning in language models through multiagent debate

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. Improv- ing factuality and reasoning in language models through multiagent debate. InForty-first International Conference on Machine Learning, 2023

    2023

  3. [11]

    Large language model based multi-agents: a survey of progress and challenges

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: a survey of progress and challenges. InProceedings of the Thirty-Third International Joint Conference on Artificial Inte...

    2024

  4. [12]

    Improving llm reasoning with multi-agent tree-of-thought validator agent.arXiv preprint arXiv:2409.11527, 2024

    Fatemeh Haji, Mazal Bethany, Maryam Tabar, Jason Chiang, Anthony Rios, and Peyman Najafirad. Improving llm reasoning with multi-agent tree-of-thought validator agent.arXiv preprint arXiv:2409.11527, 2024

    2024 arXiv

  5. [13]

    Llm multi-agent systems: Challenges and open problems.arXiv preprint arXiv:2402.03578, 2024

    Shanshan Han, Qifan Zhang, Yuhang Yao, Weizhao Jin, Zhaozhuo Xu, and Chaoyang He. Llm multi-agent systems: Challenges and open problems.arXiv preprint arXiv:2402.03578, 2024

    2024 arXiv

  6. [14]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. InInternational Conference on Learning Representations

  7. [15]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021

    2021 arXiv

  8. [16]

    Genetic algorithms.Scientific american, 267(1):66–73, 1992

    John H Holland. Genetic algorithms.Scientific american, 267(1):66–73, 1992. 10

    1992

  9. [17]

    Metagpt: Meta programming for a multi-agent collaborative framework

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al. Metagpt: Meta programming for a multi-agent collaborative framework. InThe Twelfth International Conference on Learning Representations

  10. [18]

    Optimizing niche center for multimodal optimization problems.IEEE Transactions on Cybernetics, 53(4):2544–2557, 2023

    Yi Jiang, Zhi-Hui Zhan, Kay Chen Tan, and Jun Zhang. Optimizing niche center for multimodal optimization problems.IEEE Transactions on Cybernetics, 53(4):2544–2557, 2023

    2023

  11. [19]

    Particle swarm optimization

    James Kennedy and Russell Eberhart. Particle swarm optimization. InProceedings of ICNN’95- international conference on neural networks, volume 4, pages 1942–1948. ieee, 1995

    1942

  12. [20]

    Large language models are zero-shot reasoners.Advances in neural information processing systems, 35:22199–22213, 2022

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners.Advances in neural information processing systems, 35:22199–22213, 2022

    2022

  13. [21]

    Camel: Communicative agents for" mind" exploration of large language model society.Advances in Neural Information Processing Systems, 36:51991–52008, 2023

    Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for" mind" exploration of large language model society.Advances in Neural Information Processing Systems, 36:51991–52008, 2023

    2023

  14. [22]

    Encouraging divergent thinking in large language models through multi- agent debate

    Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. Encouraging divergent thinking in large language models through multi- agent debate. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Pro...

    2024

  15. [23]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  16. [24]

    Differential evolution for multimodal optimization with species by nearest-better clustering.IEEE Transactions on Cybernetics, 51(2):970–983, 2021

    Xin Lin, Wenjian Luo, and Peilan Xu. Differential evolution for multimodal optimization with species by nearest-better clustering.IEEE Transactions on Cybernetics, 51(2):970–983, 2021

    2021

  17. [25]

    Are your llms capable of stable reasoning?arXiv preprint arXiv:2412.13147, 2024

    Junnan Liu, Hongwei Liu, Linchen Xiao, Ziyi Wang, Kuikun Liu, Songyang Gao, Wenwei Zhang, Songyang Zhang, and Kai Chen. Are your llms capable of stable reasoning?arXiv preprint arXiv:2412.13147, 2024

    2024 arXiv

  18. [26]

    Hybridizing niching, particle swarm optimization, and evolution strategy for multimodal optimization.IEEE Transactions on Cybernetics, 52(7):6707–6720, 2022

    Wenjian Luo, Yingying Qiao, Xin Lin, Peilan Xu, and Mike Preuss. Hybridizing niching, particle swarm optimization, and evolution strategy for multimodal optimization.IEEE Transactions on Cybernetics, 52(7):6707–6720, 2022

    2022

  19. [27]

    American invitational mathematics examination - aime

    MAA. American invitational mathematics examination - aime. InAmerican Invitational Mathematics Examination - AIME 2024, February 2024

    2024

  20. [28]

    American invitational mathematics examination - aime

    MAA. American invitational mathematics examination - aime. InAmerican Invitational Mathematics Examination - AIME 2025, February 2025

    2025

  21. [29]

    Samir W. Mahfoud. Crowding and preselection revisited. InParallel Problem Solving From Nature, pages 27–36. Elsevier, 1992

    1992

  22. [30]

    A clearing procedure as a niching method for genetic algorithms

    Alain Pétrowski. A clearing procedure as a niching method for genetic algorithms. InProceed- ings of IEEE International Conference on Evolutionary Computation, pages 798–803. IEEE, 1996

    1996

  23. [31]

    Niching the CMA-ES via nearest-better clustering

    Mike Preuss. Niching the CMA-ES via nearest-better clustering. InProceedings of the 12th Annual Conference Companion on Genetic and Evolutionary Computation, pages 1711–1718, 2010

    2010

  24. [32]

    Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces.Journal of global optimization, 11:341–359, 1997

    Rainer Storn and Kenneth Price. Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces.Journal of global optimization, 11:341–359, 1997

    1997

  25. [33]

    Multi-agent collaboration mechanisms: A survey of llms.arXiv preprint arXiv:2501.06322, 2025

    Khanh-Tung Tran, Dung Dao, Minh-Duong Nguyen, Quoc-Viet Pham, Barry O’Sullivan, and Hoang D Nguyen. Multi-agent collaboration mechanisms: A survey of llms.arXiv preprint arXiv:2501.06322, 2025

    2025 arXiv

  26. [34]

    CRC press, 1994

    Matt P Wand and M Chris Jones.Kernel Smoothing. CRC press, 1994. 11

    1994

  27. [35]

    Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models.arXiv preprint arXiv:2305.04091, 2023

    Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models.arXiv preprint arXiv:2305.04091, 2023

    2023 arXiv

  28. [36]

    Rethinking the bounds of llm reasoning: Are multi-agent discussions the key?arXiv preprint arXiv:2402.18272, 2024

    Qineng Wang, Zihao Wang, Ying Su, Hanghang Tong, and Yangqiu Song. Rethinking the bounds of llm reasoning: Are multi-agent discussions the key?arXiv preprint arXiv:2402.18272, 2024

    2024 arXiv

  29. [37]

    Self-consistency improves chain of thought reasoning in language models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. InThe Eleventh International Conference on Learning Representations

  30. [38]

    Yen, and Wu Song

    Yong Wang, Han-Xiong Li, Gary G. Yen, and Wu Song. MOMMOP: Multiobjective opti- mization for locating multiple optimal solutions of multimodal optimization problems.IEEE Transactions on Cybernetics, 45(4):830–843, 2015

    2015

  31. [39]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  32. [40]

    A penalty-based differential evolution for multimodal optimization.IEEE Transactions on Cybernetics, 52(7):6024–6033, 2022

    Zhifang Wei, Weifeng Gao, Genghui Li, and Qingfu Zhang. A penalty-based differential evolution for multimodal optimization.IEEE Transactions on Cybernetics, 52(7):6024–6033, 2022

    2022

  33. [41]

    Autogen: Enabling next-gen llm applications via multi-agent conversations

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. Autogen: Enabling next-gen llm applications via multi-agent conversations. InFirst Conference on Language Modeling

  34. [42]

    An alternative way of evolutionary multimodal optimization: density-based population initialization strategy

    Peilan Xu, Wenjian Luo, Jiafei Xu, Yingying Qiao, Jiajia Zhang, and Naijie Gu. An alternative way of evolutionary multimodal optimization: density-based population initialization strategy. Swarm and Evolutionary Computation, 67:100971, 2021

    2021

  35. [43]

    Tree of thoughts: Deliberate problem solving with large language models.Ad- vances in neural information processing systems, 36:11809–11822, 2023

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models.Ad- vances in neural information processing systems, 36:11809–11822, 2023

    2023

  36. [44]

    Exchange-of-thought: Enhancing large language model capabilities through cross-model communication

    Zhangyue Yin, Qiushi Sun, Cheng Chang, Qipeng Guo, Junqi Dai, Xuanjing Huang, and Xipeng Qiu. Exchange-of-thought: Enhancing large language model capabilities through cross-model communication. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Proce...

    2023

  37. [45]

    If multi-agent debate is the answer, what is the question?arXiv preprint arXiv:2502.08788, 2025

    Hangfan Zhang, Zhiyao Cui, Xinrun Wang, Qiaosheng Zhang, Zhen Wang, Dinghao Wu, and Shuyue Hu. If multi-agent debate is the answer, what is the question?arXiv preprint arXiv:2502.08788, 2025

    2025 arXiv

  38. [46]

    The lessons of developing process reward models in mathematical reasoning.arXiv preprint arXiv:2501.07301, 2025

    Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The lessons of developing process reward models in mathematical reasoning.arXiv preprint arXiv:2501.07301, 2025

    2025 arXiv

  39. [47]

    Automatic chain of thought prompting in large language models

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. InThe Eleventh International Conference on Learning Representa- tions

  40. [48]

    Least-to-most prompting enables complex reasoning in large language models

    Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V Le, et al. Least-to-most prompting enables complex reasoning in large language models. InThe Eleventh International Conference on Learning Repres...

  41. [49]

    Reso: A reward-driven self- organizing llm-based multi-agent system for reasoning tasks.arXiv preprint arXiv:2503.02390, 2025

    Heng Zhou, Hejia Geng, Xiangyuan Xue, Zhenfei Yin, and Lei Bai. Reso: A reward-driven self- organizing llm-based multi-agent system for reasoning tasks.arXiv preprint arXiv:2503.02390, 2025

    2025 arXiv

  42. [50]

    assistant agent

    Ying Zhu, Peilan Xu, Jiahao Huang, Xin Lin, and Wenjian Luo. Density-assisted evolutionary dynamic multimodal optimization.ACM Transactions on Evolutionary Learning. 12 A Supplementary Related Work A.1 Multi-Agent Frameworks Large Language Model (LLM)-based multi-agent systems...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.