Pith. sign in

REVIEW 4 major objections 6 minor 4 cited by

Multi-Agent System for Cosmological Parameter Analysis

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A multi-agent LLM system wrote and executed a full cosmological MCMC analysis without any human-written code, reproducing published ACT DR6 lensing constraints.

desk verdict A transparent proof-of-concept that an LLM multi-agent system can reproduce an ACT DR6 lensing MCMC without human-written code, but the evidence rests on a single development-task session with qualitative comparison. read the letter →

arxiv 2412.00431 v2 pith:YHSFMRDT submitted 2024-11-30 astro-ph.IM astro-ph.COphysics.comp-phphysics.data-an

Andrew Laverick , Kristen Surrao , Inigo Zubeldia , Boris Bolliet , Miles Cranmer , Antony Lewis , Blake Sherwin , Julien Lesgourgues This is my paper · ORCID
classification astro-ph.IMastro-ph.COphysics.comp-phphysics.data-an
keywords multi-agentsystemslargelanguagemodelsretrieval-augmentedgenerationscientificworkflowautomationMarkovChainMonteCarlocosmologicalparameterestimationACTDR6lensingresearchsoftware
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper reports that a multi-agent system built from large language models, retrieval-augmented generation, and locally executed code can take over an entire state-of-the-art cosmological analysis without any human-written code. The central demonstration is a full Markov Chain Monte Carlo parameter estimation of the ACT DR6 CMB lensing likelihood that converged in about forty minutes, cost $1.55, and matched the published contours. The authors argue this points toward a more general strategy for automating quantitative scientific workflows, and they show preliminary evidence that the system also handles research software it was not developed on. If this is right, the hard part of reproducing or redoing a cosmological analysis—assembling documentation, writing configuration files, and executing the pipeline—could shrink from hours or days to minutes, and the same architecture could transfer to other quantitative sciences.

What carries the argument

The system is a multi-agent architecture with three agent types: manager agents (planner, chat manager, admin) that split a task into steps and control transitions; coder agents (engineer and executor) that write and run Python code; and RAG agents that retrieve context from vectorized databases of papers, tutorials, and past session summaries. A memory agent stores summaries of past sessions to avoid repeating mistakes. Controllability comes from low temperature and TopP settings, restricted allowed transitions, and mandatory human feedback at each step. The central mechanism is the planner-driven loop: plan, retrieve, write, execute, check, iterate—with a human only approving or redirecting.

What would settle it

A decisive test would be to run cmbagent on a held-out cosmological analysis (for example, parameter estimation with the Planck PR4 lensing likelihood and a different theory code) and count how many independent sessions produce converged chains and correct contours with no human-written code; the central claim of general automation is false if most sessions fail or require substantial human code fixes.

Watch

Extended reading notes

Core claim

Within a single session, cmbagent wrote and executed the full MCMC analysis for the ACT DR6 CMB lensing likelihood, with no human-written code, and reproduced the results presented in Madhavacheril et al. (2024). The contours overlap almost perfectly, with statistically insignificant differences, and the full analysis took about 40 minutes at a cost of $1.55. The authors also report successful results on two tasks outside the core development: computing CMB power spectra for values of the parameter fEDE in classy_sz, and evaluating a cosmocnc galaxy cluster number-count likelihood as a function of mass bias. These successes are presented as evidence that the multi-agent architecture can generalize beyond the specific pipeline it was built around.

Load-bearing premise

The claim of general usefulness rests on the assumption that the main task's success is not just the result of instructions, retrieval content, and human feedback tuned to that exact benchmark during development, since the paper itself notes that the main task is the same one used to develop the system.

Editorial extensions

If this is right

  • A full cosmological parameter analysis can in principle be handed to an agentic system that writes the configuration, runs the MCMC, and produces posterior contours without human-written code.
  • Reproducing a published likelihood analysis becomes a session of about forty minutes and roughly $1.50 in API cost on a laptop, rather than days of manual code assembly.
  • The same planner/RAG/coder architecture can be pointed at research software with sparse documentation (classy_sz, cosmocnc) and still produce working code for new parameters and likelihood evaluations.
  • Because the system logs plans and session summaries, an entire analysis becomes an auditable record of prompts, code, and outputs, useful as a cross-check on human pipelines.
  • The cost and token counts are small enough that repeated agentic runs are feasible for exploratory work, not just one-off demonstrations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the architecture transfers, agentic systems could serve as low-cost analysis apprentices that draft complete pipelines for junior researchers, with a senior human validating each step.
  • The strongest test of general capability would be running cmbagent on a likelihood or dataset never seen during development, and measuring success rate over many independent runs, since LLM behavior is stochastic even at low temperature.
  • A natural next benchmark is to let the system propose its own pipeline changes and measure whether it finds a faster or higher-fidelity configuration than the human baseline.
  • The memory agent's accumulation of past session summaries could be studied as a within-system learning signal: comparing first-run versus later-run token counts and error rates on similar tasks would quantify how much experience the system actually retains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript presents cmbagent, a multi-agent LLM system built on the autogen/ag2 framework, in which RAG agents, coder agents, planner/manager agents, and a human 'admin' interact to retrieve documentation, write Python code, execute it, and iterate. The main demonstration is a single end-to-end session in which the system wrote and ran a cobaya MCMC analysis of the ACT DR6 CMB lensing likelihood using classy_sz as the theory code, obtaining posterior contours that the authors state overlap the ACT collaboration chains. Two additional 'generalization' demonstrations use classy_sz to compute fEDE power spectra and cosmocnc to evaluate a cluster-count likelihood as a function of mass bias. The paper emphasizes its work-in-progress status, discusses cost and token usage, and releases code at GitHub. The authors explicitly note that the main task was the same task used to develop the system and that the current system relies on human feedback at every step.

Significance. The paper's value, if the claims are supported, is as an early proof-of-concept that a carefully orchestrated multi-agent LLM system can assemble and execute a real cosmological inference pipeline under human supervision. The strengths are concrete: the system is open source, the main task is a genuine state-of-the-art likelihood, session transcripts are linked, and the authors are candid about limitations such as the risk of LLMs producing plausible-looking but physically incorrect responses. However, because the headline result rests on a single session and a qualitative contour comparison, the current evidence is anecdotal; it does not establish reliability or automation in a measurable sense. The generalization tests are also not benchmarked against independent references, so their evidentiary weight is limited.

major comments (4)
  1. [Section 3.1, Figure 2] The claim that the cmbagent contours 'overlap almost perfectly' and differ by 'statistically insignificant' amounts is not supported by any quantitative comparison. I request a table or text reporting, for each cosmological parameter sampled by both chains, the posterior mean, the 68% credible interval, and the difference in units of the combined uncertainty, or an equivalent sample-based distance statistic. Without such a metric, a reader cannot judge whether the reproduction is at the precision claimed, particularly because the paper attributes the residual difference to emulator precision settings; that explanation should be tested by quantifying the emulator error or by recomputing one chain with matched precision settings.
  2. [Section 2.3 and Section 3.1] The main demonstration is a single session on the very task used to develop the system. The manuscript states in Section 2.3 that 'finding instructions that yielded a good behavior ... required a significant amount of prompt engineering' and in Section 3.1 that the result 'is not a demonstration of the usefulness of the system beyond that specific task.' A single successful in-sample run does not constrain the failure rate of the pipeline, which is the quantity needed to assess any automation claim. Please report repeated independent runs of the main task, including the number of runs that reached convergence, the number that required human intervention, and the scatter in the resulting parameter constraints.
  3. [Section 3.2 and Appendix A] The two generalization demonstrations are not validated quantitatively against independent references. For the fEDE spectra in Figure 4, no reference spectra or accuracy metric is shown, so the reader cannot tell whether the code was correct or merely plausible. For the cosmocnc likelihood in Figure 3 and Appendix A, the only check is that the peak lies near the input value bias_sz = 0.8; no comparison with a human-generated likelihood curve, no numerical value of the peak, and no tolerance is given. In addition, both software packages are authored by members of the same team, so these tests do not demonstrate the ability to learn unfamiliar external software. I request at least one held-out task with an independent, pre-existing pipeline as ground truth, with a quantitative agreement metric.
  4. [Section 2.2, memory RAG agent] The paper does not state whether the memory RAG agent's database contained session summaries from earlier development runs when the reported main-task session was executed. Because the memory agent is designed to draw on 'past tasks' and 'past material,' a pre-populated memory could make the reported success partially a retrieval result rather than a fresh agentic solution. Please document the memory database state at the start of each reported session and specify which memory entries, if any, were retrieved during the main-task and generalization runs.
minor comments (6)
  1. [Abstract] The word 'etrieval-augmented' should be 'retrieval-augmented'.
  2. [Section 1] The phrase 'we emphasize to our strategy' should be 'we emphasize that our strategy'.
  3. [Section 2.3] The convergence diagnostic is named 'Gellman-Rubin'; this should be 'Gelman-Rubin'.
  4. [Figure 3] The title 'Simons Obsevatory' should be 'Simons Observatory'.
  5. [Section 2.3 and Appendix A] The manuscript would benefit from specifying the exact software versions and commit hashes of cmbagent, cobaya, classy_sz, and cosmocnc, as well as the OpenAI model snapshot used, for each reported session; the current text only names gpt-4o-2024-05-13.
  6. [Section 2.3] Setting temperature and TopP to small values is described as a way to make the system 'as deterministic and controllable as possible,' but these parameters do not guarantee determinism; consider rewording to 'low stochasticity' and reporting a test of run-to-run variability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central reproduction is benchmarked against external ACT chains, and the admitted development-on-the-same-task caveat is a generalization limitation, not a derivation loop.

full rationale

The paper's central claim is that the cmbagent multi-agent system reproduced the ACT DR6 CMB lensing cosmological parameter constraints from Madhavacheril et al. (2024) within a single session and without human-written code. This claim is checked against an external benchmark: the original ACT chains downloaded from the ACT repository ("the original chains downloaded from ACT repository are in blue"), and the agreement is stated as "statistically insignificant differences" with the residual difference attributed to emulator precision settings. The posterior is obtained by running the actual ACT likelihood and cobaya/classy_sz, not by fitting to the target contours, so the result is not equivalent to the input by construction. The paper explicitly acknowledges that the main task is the same one used to develop the system and therefore "not a demonstration of the usefulness of the system beyond that specific task"; that is a prompt-engineering overfitting / generalizability caveat, not a circular derivation step. The generalization demonstrations use classy_sz and cosmocnc, research software co-authored by the present team, but the task is to execute that software and retrieve its documentation, and the cosmocnc likelihood test is validated against a known injected mass-bias value ("The true input value of the mass bias is shown as the dashed line; it is consistent with the likelihood constraint obtained by cmbagent"). No equation, parameter, or fitted quantity is renamed as a prediction, and no load-bearing argument reduces to a self-citation. The main risks (single session, same-task development, qualitative contour comparison) belong under soundness and selection bias, not circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new physical entities or fitted cosmological parameters are introduced; the paper proposes an agent architecture rather than a physical model. Its free parameters are LLM hyperparameters, and its key axioms concern LLM reliability, documentation fidelity, and the correctness of the research software used in the tests.

free parameters (2)
  • LLM temperature = 1e-6
    Chosen by hand in Section 2.3 to reduce stochasticity; not fitted to data, but a free design choice affecting reproducibility.
  • TopP = 0.1
    Chosen by hand alongside temperature in Section 2.3 to control output variability.
assumptions (4)
  • domain assumption GPT-4o with the given prompts, RAG content, and human feedback will write correct Python code for the target tasks.
    The entire demonstration depends on LLM code-generation reliability; only one successful session is shown in Section 3.1.
  • domain assumption The RAG databases correctly capture the ACT DR6 likelihood, cobaya, classy_sz, and cosmocnc usage.
    The agents rely on retrieved documentation, and the paper acknowledges retrieval of scientific documentation with complex tables and equations is difficult in Section 4.
  • domain assumption The ACT DR6 official chains are a valid external ground truth for the reproduction.
    Used as the benchmark in Figure 2; this is standard practice in cosmology.
  • domain assumption The classy_sz emulators and cosmocnc likelihood produce the correct physical outputs used in the generalization tests.
    These packages are co-developed by the authors and are not independently validated within this paper, as described in Sections 2.1 and 3.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-Agent System for Cosmological Parameter Analysis." pith.science (2026). https://pith.science/paper/YHSFMRDT

@misc{pith2026241200431,
  author       = {Pith},
  title        = {Pith review of: Multi-Agent System for Cosmological Parameter Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YHSFMRDT}},
  note         = {Machine review of arXiv:2412.00431}
}
read the original abstract

Multi-agent systems (MAS) utilizing multiple Large Language Model agents with Retrieval Augmented Generation and that can execute code locally may become beneficial in cosmological data analysis. Here, we illustrate a first small step towards AI-assisted analyses and a glimpse of the potential of MAS to automate and optimize scientific workflows in Cosmology. The system architecture of our example package, that builds upon the autogen/ag2 framework, can be applied to MAS in any area of quantitative scientific research. The particular task we apply our methods to is the cosmological parameter analysis of the Atacama Cosmology Telescope lensing power spectrum likelihood using Monte Carlo Markov Chains. Our work-in-progress code is open source and available at https://github.com/CMBAgents/cmbagent.

Figures

Figures reproduced from arXiv: 2412.00431 by the authors.

Figure 1
Figure 1. Overview of system hierarchy and transitions. Manager agents are shown in salmon, coder [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Reproducing of pipelines for the ACT DR6 CMB lensing cosmological parameter con [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Number-count likelihood for a Simons-Observatory-like simulated catalogue as a function [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: CMB power spectra for ten values of the cosmological parameter [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Agentic Re-Casting using Agentic Re-Simulations

    hep-ph 2026-07 conditional novelty 6.0 of 10

    An agentic AI system with a physicist in the loop re-casts an ATLAS ttZ measurement into a global top-quark SMEFT fit and recovers injected coloron Wilson coefficients in a repeatable benchmark.

  2. DeepInflation: an AI agent for research and model discovery of inflation

    astro-ph.CO 2026-01 conditional novelty 6.0 of 10

    An LLM agent with symbolic regression finds simple inflation potentials that match target CMB observables, but the outputs are fitted to the targets rather than independently predicted.

  3. Bridging Literature and the Universe Via A Multi-Agent Large Language Model System

    astro-ph.IM 2025-07 conditional novelty 6.0 of 10

    A multi-agent large language model system extracts cosmological simulation parameters from published papers and generates valid MP-GADGET configuration files, achieving 98.7% micro-F1 on a new 41-simulation benchmark.

  4. A Comprehensive Survey of Deep Research: Systems, Methodologies, and Applications

    cs.AI 2025-06 conditional novelty 4.0 of 10

    A survey of 80+ Deep Research systems that proposes a four-layer taxonomy (foundation models, tool use, planning, synthesis) and compares commercial and open-source implementations.

Reference graph

Works this paper leans on

43 extracted references · 5 canonical work pages · cited by 4 Pith papers

  1. [1]

    Wu , author G

    author Q. Wu , author G. Bansal , author J. Zhang , author Y. Wu , author B. Li , author E. Zhu , author L. Jiang , author X. Zhang , author S. Zhang , author J. Liu , et al. , title Autogen: Enabling next-gen llm applications via multi-agent conversation ( year 2023 ), 2308.08155

    arXiv 2023

  2. [2]

    Touvron , author T

    author H. Touvron , author T. Lavril , author G. Izacard , author X. Martinet , author M.-A. Lachaux , author T. Lacroix , author B. Rozi \`e re , author N. Goyal , author E. Hambro , author F. Azhar , et al. , title LLaMA: Open and Efficient Foundation Language Models ( year 2023 ), 2302.13971

    arXiv 2023

  3. [3]

    author T. B. Brown , author B. Mann , author N. Ryder , author M. Subbiah , author J. Kaplan , author P. Dhariwal , author A. Neelakantan , author P. Shyam , author G. Sastry , author A. Askell , et al. , title Language Models are Few-Shot Learners ( year 2020 ), 2005.14165

    arXiv 2020

  4. [4]

    Achiam , author S

    author OpenAI , author J. Achiam , author S. Adler , author S. Agarwal , author L. Ahmad , author I. Akkaya , author F. Leoni Aleman , author D. Almeida , author J. Altenschmidt , author S. Altman , et al. , title GPT-4 Technical Report ( year 2023 ), 2303.08774

    arXiv 2023

  5. [5]

    Transformers for scientific data: a pedagogical review for astronomers

    author D. Tanoglidis , author B. Jain , and author H. Qu , title Transformers for scientific data: a pedagogical review for astronomers ( year 2023 ), 2310.12069

    work page Pith review arXiv 2023

  6. [6]

    Ghafarollahi and author M

    author A. Ghafarollahi and author M. J. Buehler , title SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning ( year 2024 ), 2409.05556

    arXiv 2024

  7. [7]

    author M. J. Buehler , title Accelerating scientific discovery with generative knowledge extraction, graph-based representation, and multimodal intelligent graph reasoning ( year 2024 ), ://iopscience.iop.org/article/10.1088/2632-2153/ad7228

    doi:10.1088/2632-2153/ad7228 2024

  8. [8]

    Sun , author Y.-S

    author Z. Sun , author Y.-S. Ting , author Y. Liang , author N. Duan , author S. Huang , and author Z. Cai , title Interpreting Multi-band Galaxy Observations with Large Language Model-Based Agents ( year 2024 ), 2409.14807

    arXiv 2024

Show all 43 references
  1. [9]

    author M. S. Madhavacheril et al. ( collaboration ACT ), title The Atacama Cosmology Telescope: DR6 Gravitational Lensing Map and Cosmological Parameters ( year 2024 ), 2304.05203

    2024 arXiv

  2. [10]

    author F. J. Qu et al. ( collaboration ACT ), title The Atacama Cosmology Telescope: A Measurement of the DR6 CMB Lensing Power Spectrum and Its Implications for Structure Growth ( year 2024 ), 2304.05202

    2024 arXiv

  3. [11]

    MacCrann , author B

    author N. MacCrann , author B. D. Sherwin , author F. J. Qu , author T. Namikawa , author M. S. Madhavacheril , author I. Abril-Cabezas , author R. An , author J. E. Austermann , author N. Battaglia , author E. S. Battistelli , et al. , title The atacama cosmology telescope: M...

    2023 arXiv

  4. [12]

    Lewis , author A

    author A. Lewis , author A. Challinor , and author A. Lasenby , title Efficient computation of CMB anisotropies in closed FRW models ( year 2000 ), astro-ph/9911177

    2000 arXiv

  5. [13]

    Blas , author J

    author D. Blas , author J. Lesgourgues , and author T. Tram , title The cosmic linear anisotropy solving system (class). part ii: Approximation schemes ( year 2011 ), 1104.2933

    2011 arXiv

  6. [14]

    Torrado and author A

    author J. Torrado and author A. Lewis , title Cobaya: Code for Bayesian Analysis of hierarchical physical models ( year 2021 ), 2005.05290

    2021 arXiv

  7. [15]

    Lewis , title GetDist: a Python package for analysing Monte Carlo samples ( year 2019 ), 1910.13970

    author A. Lewis , title GetDist: a Python package for analysing Monte Carlo samples ( year 2019 ), 1910.13970

    2019 arXiv

  8. [16]

    Bolliet et al

    author B. Bolliet et al. , title class\_sz I: Overview ( year 2024 ), 2310.18482

    2024 arXiv

  9. [17]

    author I. n. Zubeldia , author J.-B. Melin , author J. Chluba , and author R. Battye , title The Planck SZiFi catalogues: a new set of Planck catalogues of Sunyaev-Zeldovich-detected galaxy clusters ( year 2024 ), 2408.06189

    2024 arXiv

  10. [18]

    Abadi , author P

    author M. Abadi , author P. Barham , author J. Chen , author Z. Chen , author A. Davis , author J. Dean , author M. Devin , author S. Ghemawat , author G. Irving , author M. Isard , et al. , title TensorFlow: A system for large-scale machine learning ( year 2016 ), 1605.08695

    2016 arXiv

  11. [19]

    Spurio-Mancini , author D

    author A. Spurio-Mancini , author D. Piras , author J. Alsing , author B. Joachimi , and author M. P. Hobson , title <scp>cosmopower</scp>: emulating cosmological power spectra for accelerated bayesian inference from next-generation surveys ( year 2022 ), 2106.03846

    2022 arXiv

  12. [20]

    Bolliet , author A

    author B. Bolliet , author A. Spurio Mancini , author J. C. Hill , author M. Madhavacheril , author H. T. Jense , author E. Calabrese , and author J. Dunkley , title High-accuracy emulators for observables in CDM, N _ eff , m _ , and w cosmologies ( year 2024 ), 2303.01591

    2024 arXiv

  13. [21]

    u ttler , author M. Lewis , author W.-t. Yih , author T. Rockt \

    author P. Lewis , author E. Perez , author A. Piktus , author F. Petroni , author V. Karpukhin , author N. Goyal , author H. K \"u ttler , author M. Lewis , author W.-t. Yih , author T. Rockt \"a schel , et al. , title Retrieval-Augmented Generation for Knowledge-Intensive NLP...

    2020 arXiv

  14. [22]

    Aghanim et al

    author N. Aghanim et al. ( collaboration Planck ), title Planck 2018 results. VI. Cosmological parameters ( year 2020 ), note [Erratum: Astron.Astrophys. 652, C4 (2021)] , 1807.06209

    2021 arXiv

  15. [23]

    Carron , author M

    author J. Carron , author M. Mirmelstein , and author A. Lewis , title CMB lensing from Planck PR4 maps ( year 2022 ), 2206.07773

    2022 arXiv

  16. [24]

    Tristram et al

    author M. Tristram et al. , title Cosmological parameters derived from the final Planck data release (PR4) ( year 2024 ), 2309.10034

    2024 arXiv

  17. [25]

    Wang , author B

    author X. Wang , author B. Li , author Y. Song , author F. F. Xu , author X. Tang , author M. Zhuge , author J. Pan , author Y. Song , author B. Li , author J. Singh , et al. , title Openhands: An open platform for ai software developers as generalist agents ( year 2024 ), 2407.16741

    2024 arXiv

  18. [26]

    author M. D. Skarlinski , author S. Cox , author J. M. Laurent , author J. D. Braza , author M. Hinks , author M. J. Hammerling , author M. Ponnapati , author S. G. Rodriques , and author A. D. White , title Language agents achieve superhuman synthesis of scientific knowledge ...

    2024 arXiv

  19. [27]

    Chiang , author L

    author W.-L. Chiang , author L. Zheng , author Y. Sheng , author A. N. Angelopoulos , author T. Li , author D. Li , author H. Zhang , author B. Zhu , author M. Jordan , author J. E. Gonzalez , et al. , title Chatbot arena: An open platform for evaluating llms by human preferen...

    2024 arXiv

  20. [28]

    de Haan , title cosmosage: A Natural-Language Assistant for Cosmologists ( year 2024 ), 2407.04420

    author T. de Haan , title cosmosage: A Natural-Language Assistant for Cosmologists ( year 2024 ), 2407.04420

    2024 arXiv

  21. [29]

    author T. D. Nguyen et al. , title AstroLLaMA: Towards Specialized Foundation Models in Astronomy ( year 2023 ), 2309.06126

    2023 arXiv

  22. [30]

    Ting , author T

    author Y.-S. Ting , author T. D. Nguyen , author T. Ghosal , author R. Pan , author H. Arora , author Z. Sun , author T. de Haan , author N. Ramachandra , author A. Wells , author S. Madireddy , et al. , title Astromlab 1: Who wins astronomy jeopardy!? ( year 2024 ), 2407.11194

    2024 arXiv

  23. [31]

    Pan , author T

    author R. Pan , author T. Dung Nguyen , author H. Arora , author A. Accomazzi , author T. Ghosal , and author Y.-S. Ting , title AstroMLab 2: AstroLLaMA-2-70B Model and Benchmarking Specialised LLMs for Astronomy ( year 2024 ), 2409.19750

    2024 arXiv

  24. [32]

    de Haan , author Y.-S

    author T. de Haan , author Y.-S. Ting , author T. Ghosal , author T. D. Nguyen , author A. Accomazzi , author A. Wells , author N. Ramachandra , author R. Pan , and author Z. Sun , title Astromlab 3: Achieving gpt-4o level performance in astronomy with a specialized 8b-paramet...

    2024 arXiv

  25. [33]

    Canese , author G

    author L. Canese , author G. C. Cardarilli , author L. Di Nunzio , author R. Fazzolari , author D. Giardino , author M. Re , and author S. Spanò , title Multi-agent reinforcement learning: A review of challenges and applications ( year 2021 ), ://www.mdpi.com/2076-3417/11/11/4948

    2021

  26. [34]

    Qi , author M

    author Z. Qi , author M. Ma , author J. Xu , author L. Lyna Zhang , author F. Yang , and author M. Yang , title Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers ( year 2024 ), 2408.06195

    2024 arXiv

  27. [35]

    Cheng , author A

    author C.-A. Cheng , author A. Nie , and author A. Swaminathan , title Trace is the next autodiff: Generative optimization with rich feedback, execution traces, and llms ( year 2024 ), 2406.16218

    2024 arXiv

  28. [36]

    Chen , author J

    author W. Chen , author J. Yuan , author C. Qian , author C. Yang , author Z. Liu , and author M. Sun , title Optima: Optimizing effectiveness and efficiency for llm-based multi-agent system ( year 2024 ), 2410.08115

    2024 arXiv

  29. [37]

    Lu , author C

    author C. Lu , author C. Lu , author R. Tjarko Lange , author J. Foerster , author J. Clune , and author D. Ha , title The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery ( year 2024 ), 2408.06292

    2024 arXiv

  30. [38]

    Papoudakis , author F

    author G. Papoudakis , author F. Christianos , author L. Sch \"a fer , and author S. V. Albrecht , title Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks ( year 2020 ), 2006.07869

    2020 arXiv

  31. [39]

    Bettini , author A

    author M. Bettini , author A. Prorok , and author V. Moens , title BenchMARL: Benchmarking Multi-Agent Reinforcement Learning ( year 2023 ), 2312.01472

    2023 arXiv

  32. [40]

    Schulman , author F

    author J. Schulman , author F. Wolski , author P. Dhariwal , author A. Radford , and author O. Klimov , title Proximal policy optimization algorithms ( year 2017 ), 1707.06347 , ://arxiv.org/abs/1707.06347

    2017 arXiv

  33. [41]

    Yang and author J

    author Y. Yang and author J. Wang , title An overview of multi-agent reinforcement learning from game theoretical perspective ( year 2021 ), 2011.00583 , ://arxiv.org/abs/2011.00583

    2021 arXiv

  34. [42]

    Wen , author Z

    author M. Wen , author Z. Wan , author W. Zhang , author J. Wang , and author Y. Wen , title Reinforcing language agents via policy optimization with action decomposition ( year 2024 ), 2405.15821 , ://arxiv.org/abs/2405.15821

    2024 arXiv

  35. [43]

    author S. V. Albrecht , author F. Christianos , and author L. Sch\"afer , title Multi-Agent Reinforcement Learning: Foundations and Modern Approaches ( publisher MIT Press , year 2024 ), ://www.marl-book.com

    2024

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.