Continuous Improvement and Parallel Autonomous Exploration: An LLM-Agent Framework for Searching Large Solution Spaces

Andre Rusli; Dulmini Hettiarachchi; Julio Christian Young; Sho Akiyama

REVIEW 3 major objections 3 minor 30 references

Continuous Improvement and Parallel Autonomous Exploration: An LLM-Agent Framework for Searching Large Solution Spaces

T0 review · 3 major / 3 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A withheld-test leaderboard can reward autonomous LLM agents, and running them in parallel under that reward broadens the explored region of a large solution space.

desk verdict Honest, well-scoped framework paper whose central 'parallel escape' claim is confounded and explicitly unproven; worth a serious referee for the framework itself. read the letter →

arxiv 2608.04341 v1 pith:EI5U6RQM submitted 2026-08-05 cs.MA

Dulmini Hettiarachchi , Andre Rusli , Julio Christian Young , Sho Akiyama This is my paper · ORCID

classification cs.MA

keywords multi-agentsystemsLLMagentsautonomousexplorationcontinuousimprovementretrievaloptimizationselectivepredictionleaderboardrewardproduct-to-catalogmatching

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a leaderboard scored on withheld test data can serve as a reward signal for autonomous LLM agents, and that running several such agents in parallel under that shared signal widens the explored region of a large solution space. A single seeded agent keeps refining its initial dense-embedding approach, while parallel agents branch into non-embedding methods. On a product-to-catalog matching task, best qualified coverage rises from a 33.3% baseline to 47.8–57.4% with one agent and 62.8–69.4% with five. The authors position the framework itself, not the specific numbers, as the contribution.

What carries the argument

The load-bearing object is the leaderboard: a withheld-test scoring API that returns per-category precision and coverage, checks qualification at $\ge 95\%$ Precision@1, ranks qualified submissions by average coverage, and appends results to a persistent ledger. Agents see scores and brief approach summaries but never test labels or each other's code. Around it runs a continuous loop—analyze, survey, hypothesize, implement, self-evaluate on train, submit, observe score, revise—plus an automated moderator limited to logistics. The selective-prediction framing (maximize coverage subject to $\ge 95\%$ Precision@1 in every category) turns the task into one with a large, category-structured solution space where no single method dominates.

What would settle it

Run a single agent with no seeded baseline, the same five-hour budget, and a submission count matched to the five-agent runs (about 550 submissions). If it reaches the same non-embedding methods and roughly 69% qualified coverage, then parallel independence is not needed for the escape and the central claim is falsified.

Watch

Extended reading notes

Core claim

The central claim is that two mechanisms—a continuous-improvement reward loop and a parallel autonomous exploration substrate—let LLM agents search large applied solution spaces without a human in the loop. The reward loop alone works with one agent: by resubmitting scored predictions, the agent beats the baseline substantially. The parallel mechanism changes the search qualitatively: across all runs, single agents stayed inside the seeded embedding paradigm, while five-agent runs reached string-similarity and TF-IDF methods the single agents never tried. The paper reports the coverage figures as descriptive ranges, not a causal scaling law, because agent count co-varies with total submissions, GPU contention, and shared leaderboard visibility.

Load-bearing premise

The load-bearing premise is that the qualitative difference between single and parallel runs comes from parallel exploration under a shared reward, not from the far larger number of submissions, the extra compute, GPU contention, or the single shared model's own prior; the paper itself names a no-seed control as the decisive missing experiment.

Editorial extensions

If this is right

Even one agent, resubmitting against the leaderboard, improves best qualified coverage from 33.3% to 47.8–57.4% over the baseline without any human tuning.
Five parallel agents reach 62.8–69.4%, with the entire gain over single agents concentrated in the hardest category, where single agents managed 0.8–1.9% coverage.
Parallel agents surface qualitatively different solutions—string-similarity matching and TF-IDF fusion—that a single seeded agent never instantiates, in every five-agent run.
Because the framework is task-agnostic and the only communication channel is an append-only chat log, the same substrate can be pointed at other automated-metric tasks such as search ranking, recommendation, or listing-quality checks.
Continuous operation is enforced by design: liveness heartbeats and scheduled warnings keep agents iterating instead of stopping early, so the loop's outcome does not depend on agent discretion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test of the paper's central claim would compare five parallel agents against a single agent given the same total number of submissions; if the lone agent then also reaches non-embedding methods, the diversity effect would be attributable to volume of search, not to parallel independence.
The no-seed control the authors flag could reveal whether the single agent's narrowness comes from the dense-embedding seed or from the model's own prior; that experiment would determine whether the framework needs a seed at all.
Because agents share score-and-summary signals, the leaderboard may act as an implicit diversity-preservation mechanism; an explicit test would be to toggle visibility of approach summaries and measure whether parallel agents collapse onto the same strategy.
The depth-for-breadth trade observed here suggests a tuning knob for deployment: a team could allocate fewer parallel agents when the seeded paradigm is strong, and more when the category structure rewards heterogeneous approaches.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a solid workshop-grade framework paper that is unusually honest about its limits, but the headline behavioral result—parallel agents escape the seeded paradigm while a single agent does not—is not established by the experiments, and the authors know it. The confound is not subtle: agent count, total submissions, compute, GPU contention, and visibility of other agents' approach summaries all change at once. A five-agent run is also five independent stochastic draws, so the 'escape' could just be selection. The no-seed control they call decisive is absent. That said, the paper earns credit for how it is written and scoped.

What is actually new: the configuration—independent generalist agents sharing only scores and approach descriptions under a withheld-test leaderboard, with no explicit diversity pressure—is a real, if incremental, variation on Agent-K and CoMind. The infrastructure discipline (leak checks, per-agent directories, GPU locks, autonomous logging) is described carefully and is a useful template. The single-agent result, improving from 33.3% to 47.8–57.4% qualified coverage through iterative resubmission, is a genuine existence proof of the reward loop. The parallel runs reaching non-embedding methods like TF-IDF and string matching is interesting, but as a behavioral observation, not a demonstrated property of the framework.

Soft spots, in proportion: the main one is the load-bearing comparison, as above. The paper is transparent about this and labels the numbers descriptive, so it is not overclaiming; but the Limitations section asserts the central finding is 'independent of agent count'—that overstates the evidence, since the confound remains. No code, data, or logs are released, so the framework is not independently reproducible. The approach-diversity analysis is from a single representative run, so 'within minutes' is anecdotal. These are not fatal, but they mean the paper is a proof-of-concept, not a confirmed result.

This paper is for applied researchers who want a concrete blueprint for running autonomous LLM-agent search loops with safe infrastructure. I would bring it to a reading group as an example of honest scoping. It deserves peer review—not because the headline is proven, but because the framework is well-defined and the authors are clear about what would settle the question. A good reviewer should push for a no-seed control and for releasing at least the code and logs. My verdict: conditional, with the condition being the missing control.

Referee Report

3 major / 3 minor

Summary. The paper presents an LLM-agent framework for autonomous search over large solution spaces, built on two mechanisms: a withheld-test leaderboard that serves as a reward signal for iterative refinement, and parallel execution of multiple independent agents that share only scores and high-level approach descriptions. The framework is instantiated on product-to-catalog matching in a C2C marketplace, formulated as selective prediction with a per-category precision constraint. The authors report three runs each of a single-agent and a five-agent configuration, finding best qualified coverage of 47.8–57.4% with a single agent and 62.8–69.4% with five agents, versus a 33.3% unqualified BGE-M3 baseline. They further report a qualitative pattern: single agents iterate narrowly within the seeded BGE-M3 embedding paradigm, while parallel agents adopt qualitatively different non-embedding methods such as TF-IDF re-ranking and string matching. The stated contribution is the framework itself, with the coverage figures presented as descriptive case-study evidence rather than a causal scaling law.

Significance. If the central qualitative finding is sound, the framework is a useful no-human-in-the-loop substrate for exploring applied solution spaces, and the paper's infrastructure details are a legitimate contribution: per-agent working directories, automated leak checks, an append-only chat log, GPU locking, and a logistics-only moderator are described concretely and repeated three times per configuration. The authors deserve credit for the unusually explicit limitations section and for refraining from fitted constants or derivations that would create a circularity burden. However, the main behavioral claim that parallel agents under a shared reward escape a single agent's seeded paradigm is not yet isolated from submission count, compute, stochastic draws, and model-specific priors, all of which the paper itself acknowledges. The stress-test concern therefore lands: the qualitative 'escape' result is load-bearing for the second claimed mechanism, and the current experimental design does not establish it.

major comments (3)

[§5, 'Narrow Exploration vs. Escape'] The central claim that parallel agents escape the seeded paradigm while a single agent remains narrow is confounded: the comparison varies agent count, total submissions (60/124/81 vs 274/550/301), total compute, GPU contention, and shared leaderboard visibility simultaneously. A single-agent run is one stochastic trajectory, while a five-agent run is five independent trajectories plus a shared reward channel, so the observed escape could be a selection effect of drawing five independent LLM priors or of 3–5x more submissions rather than a property of parallel coordination. The paper itself concedes this in §5: 'Because all runs use one LLM, seed, and prompt, we cannot separate this from a model-specific prior; a no-seed control is the decisive next experiment.' Since the no-seed control is absent and no control matches submission counts or isolates the shared-reward channel, the 'escape' finding is not yet established as a property of the framework.
[§5, Limitations] The statement 'Our central qualitative finding ... replicates across runs and is independent of agent count, so the limitations below do not affect it' is internally inconsistent with the very next sentence, which acknowledges that agent count co-varies with total compute, simultaneous workers, GPU contention, and shared leaderboard visibility. The central finding is precisely a single-versus-parallel comparison, so the listed confounds do bear on it. This sentence should be corrected to state which parts of the finding survive the confounds (for example, repeated qualitative observations within each configuration) and which parts require the decisive control that the authors themselves identify.
[§4.2, Table 2] The headline coverage numbers are the best qualified coverage over all submissions in a run, and the number of submissions differs by a factor of roughly 3–5 between configurations. Best-of-n maxima are biased upward with larger n, so the reported 62.8–69.4% for parallel runs versus 47.8–57.4% for single-agent runs may reflect more opportunities rather than a broader or better search. To support the coverage comparison, the authors should report final or median coverage, or compare best coverage at matched submission counts, and show per-agent trajectories (as in Figure 2 for one run) for all configurations.

minor comments (3)

[Figure 1] The caption says agents 'propose in a shared sandbox,' but §3.2 states that each agent is confined to its own working directory plus read-only shared resources; the figure caption should clarify what 'shared' means to avoid implying shared write access.
[§4.2] The text reports that 'a single agent with uncontested GPU is competitive with the parallel runs' on Trading Card categories, but Figure 2 shows only one representative 5-agent run; adding corresponding single-agent time-series would make this comparison substantiated rather than asserted.
[§4.1] The phrase 'best qualified coverage (≥95% P@1 per category)' in the abstract and §4.2 should be defined precisely: coverage is averaged over the three categories for qualified submissions, and Table 2 labels it 'Best Avg Cov.'

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation; the framework's claims are empirical and self-contained, with only a minor conceptual circularity in labeling the reward loop 'continuous improvement'.

full rationale

The paper does not derive any result from fitted parameters or from the paper's own prior claims. The leaderboard reward is measured on withheld test data, coverage improvements are reported against an external BGE-M3 baseline, and all qualitative claims rest on observed behavior in logged runs. The closest item to circularity is the framing of 'continuous improvement' as a mechanism: it is a designed property of the loop (submission, score, revise) rather than an independent discovery, but the empirical rise in qualified coverage is not forced by construction. Likewise, the claim that parallel agents 'escape' the seeded paradigm is an empirical observation; its weakness is that the single-vs-parallel comparison co-varies with total submissions, compute, GPU contention, and stochastic draws, as the paper itself concedes ('Because all runs use one LLM, seed, and prompt, we cannot separate this from a model-specific prior; a no-seed control is the decisive next experiment'). That confound is an experimental-design limitation, not circularity: the outcome is not defined in terms of the input, and no self-citation is load-bearing. Score 1 reflects the minor conceptual tautology in the framework's self-description, not a circular derivation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central empirical claims rest on the integrity of the withheld-test leaderboard, the reliability of LLM agents executing the loop, the choice of three categories, and the per-category thresholds; agent count and time budget are chosen variables. No new physical or formal entity is postulated; the moderator is a software component.

free parameters (4)

Per-category confidence thresholds = Not reported; tuned on train for 95% P@1
Baseline and agents set thresholds per category to reach the 95% P@1 operating point; coverage numbers depend directly on these thresholds, which are not fixed a priori.
Session time budget = 5 hours
Fixed by organizers; caps the number of submissions and thus the leaderboard-optimization opportunity; results may depend on it.
Number of parallel agents = 1 or 5
The central comparison variable; explicitly confounded with total compute and submission count (60-124 vs 274-550 submissions).
LLM model and prompt seed = Claude Sonnet 3.5 with one prompt template
All agents use the same LLM and seed; the paper acknowledges this prevents separating the seeded baseline from a model-specific prior.

assumptions (4)

domain assumption The withheld-test leaderboard scores are computed on ground truth agents never access
Stated in Section 3.2 with automated leak checks; integrity of the reward signal is central to the framework.
domain assumption LLM agents reliably execute the full research loop (analyze, implement, submit, revise) within the session
The behavioral results depend on agents following the prescribed loop; failures are reported only anecdotally (one early stop).
domain assumption The three selected categories are representative of the product-to-catalog solution space
Only Smartphone, Trading Card (Others), and Trading Card (Pokemon) are used; cross-task generalization is acknowledged as open.
domain assumption A single shared leaderboard score is a sufficient reward signal for useful exploration
The framework's design assumes leaderboard feedback alone drives improvement; no comparison to other reward signals is made.

invented entities (1)

Moderator agent (automated, logistics-only)
purpose: Coordinates time warnings, GPU queue, and leaderboard updates without providing technical guidance
A software component of the framework; its behavior is asserted to stay within logistics-only scope, but no logs or code are provided to verify it never drifts.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Continuous Improvement and Parallel Autonomous Exploration: An LLM-Agent Framework for Searching Large Solution Spaces." pith.science (2026). https://pith.science/paper/EI5U6RQM

@misc{pith2026260804341,
  author       = {Pith},
  title        = {Pith review of: Continuous Improvement and Parallel Autonomous Exploration: An LLM-Agent Framework for Searching Large Solution Spaces},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EI5U6RQM}},
  note         = {Machine review of arXiv:2608.04341}
}

read the original abstract

We present a framework that gives LLM agents two mechanisms for searching large solution spaces autonomously. First, a leaderboard scored on held-out data acts as a reward signal that drives each agent to refine its solutions over repeated submissions, a loop that operates even with a single agent. Second, the framework enables running many agents in parallel, fully autonomously, with no human in the loop: agents independently analyze, survey methods, implement, self-evaluate, submit, and revise, while a moderator agent handles only logistics. Running agents in parallel under the shared reward broadens the explored region of the solution space rather than refining the single seeded paradigm. We instantiate the framework on product-to-catalog matching (a core e-commerce retrieval task with a large, category-structured solution space), posed as selective prediction with a precision-coverage operating point. A single agent refines within its seeded paradigm, whereas parallel autonomous agents surface qualitatively different solutions. On this testbed, best qualified coverage (>=95% P@1 per category) reaches 47.8-57.4% with a single agent and 62.8-69.4% with five, against a 33.3% baseline. Our contribution is the framework itself: a continuous-improvement reward loop and a substrate for fully autonomous parallel exploration, backed by case-study evidence.

Figures

Figures reproduced from arXiv: 2608.04341 by the authors.

**Figure 2.** Best qualified coverage over time for one represen [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 19 canonical work pages

[1]

Daniil A Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes. 2023. Au- tonomous chemical research with large language models.Nature624, 7992 (2023), 570–578

2023
[2]

Andres M Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller. 2024. ChemCrow: Augmenting large language models with chemistry tools.Nature Machine Intelligence6, 5 (2024), 525–535

work page 2024
[3]

Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, Lilian Weng, and Aleksander Mądry. 2025. MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering. InThe Thirteenth International Conference on Learning Representations

work page 2025
[4]

Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu
[5]

Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, et al. 2024. AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors. In The Twelfth International Conference on Learning Representations

work page 2024
[6]

Tenenbaum, and Igor Mor- datch

Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mor- datch. 2024. Improving Factuality and Reasoning in Language Models through Multiagent Debate. InProceedings of the 41st International Conference on Machine Learning. PMLR, 11733–11763

work page 2024
[7]

Yonatan Geifman and Ran El-Yaniv. 2017. Selective Classification for Deep Neural Networks. InAdvances in Neural Information Processing Systems, Vol. 30

2017
[8]

Antoine Grosnit, Alexandre Maraval, James Doran, Rasul Tutunov, Wenlong Wang, Haitham Bou Ammar, et al. 2024. Large Language Models Orchestrat- ing Structured Reasoning Achieve Kaggle Grandmaster Level.arXiv preprint arXiv:2411.03562(2024)

arXiv 2024

Show all 30 references

[9]

Sirui Hong, Mingchen Zhuge, Jiaqi Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al . 2024. MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. In The Twelfth International Conference on Learning Repre...

2024
[10]

Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec. 2024. MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation. InProceed- ings of the 41st International Conference on Machine Learning. PMLR, 20271–20309

2024
[11]

Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2024. SWE-bench: Can Language Models Resolve Real-World GitHub Issues?. InThe Twelfth International Conference on Learning Representations

2024
[12]

Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2021. Billion-Scale Similarity Search with GPUs.IEEE Transactions on Big Data7, 3 (2021), 535–547

2021
[13]

Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open- Domain Question Answering. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 6769–6781

2020
[14]

Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. CAMEL: Communicative Agents for “Mind” Exploration of Large Language Model Society. InAdvances in Neural Information Processing Systems, Vol. 36

2023
[15]

Sijie Li, Weiwei Sun, Shanda Li, Ameet Talwalkar, and Yiming Yang. 2025. CoMind: Towards Community-Driven Agents for Machine Learning Engineering.arXiv preprint arXiv:2506.20640(2025)

2025
[16]

Yuliang Li, Jinfeng Li, Yoshihiko Suhara, AnHai Doan, and Wang-Chiew Tan
[17]

Ziming Li, Qianbo Zang, David Ma, Jiawei Guo, Tuney Zheng, Minghao Liu, Xinyao Niu, Yue Wang, Jian Yang, Jiaheng Liu, Wanjun Zhong, Wangchunshu Zhou, Wenhao Huang, and Ge Zhang. 2024. AutoKaggle: A Multi-Agent Frame- work for Autonomous Data Science Competitions.arXiv preprint...

2024 arXiv
[18]

Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. 2024. The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery.arXiv preprint arXiv:2408.06292(2024)

2024 arXiv
[19]

Jean-Baptiste Mouret and Jeff Clune. 2015. Illuminating search spaces by mapping elites.arXiv preprint arXiv:1504.04909(2015)

2015 arXiv
[20]

Sidharth Mudgal, Han Li, Theodoros Rekatsinas, AnHai Doan, Youngchoon Park, Ganesh Krishnan, Rohit Deep, Esteban Arcaute, and Vijay Raghavendra. 2018. Deep Learning for Entity Matching: A Design Space Exploration. InProceedings of the 2018 International Conference on Managemen...

2018
[21]

Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al. 2024. ChatDev: Communicative Agents for Software Development. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. 15174–15186

2024
[22]

Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco J R Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. 2024. Mathematical discoveries from program search with large language models.Nature625,...

2024
[23]

Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. InAdvances in Neural Information Processing Systems, Vol. 36

2023
[24]

Lei Wang, Chen Ma, Xueyang Feng, et al. 2024. A Survey on Large Language Model based Autonomous Agents.Frontiers of Computer Science18, 6 (2024), 186345

2024
[25]

Qingyun Wu, Gagan Bansal, Jieyu Zhang, et al. 2024. AutoGen: Enabling Next- Gen LLM Applications via Multi-Agent Conversation. InProceedings of the First Conference on Language Modeling (COLM)

2024
[26]

Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2024. Large Language Models as Optimizers. InThe Twelfth International Conference on Learning Representations

2024
[27]

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. InThe Eleventh International Conference on Learning Representations

2023
[28]

Qinlin Zhao, Jindong Wang, Yixuan Zhang, Yiqiao Jin, Kaijie Zhu, Hao Chen, and Xing Xie. 2024. CompeteAI: Understanding the Competition Dynamics of Large Language Model-based Agents. InProceedings of the 41st International Conference on Machine Learning. PMLR, 61092–61107

2024
[2021]

Deep Entity Matching with Pre-Trained Language Models.Proceedings of the VLDB Endowment14, 1 (2021), 50–60

2021
[2024]

InFindings of the Asso- ciation for Computational Linguistics: ACL 2024

M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. InFindings of the Asso- ciation for Computational Linguistics: ACL 2024. Association for Computational Linguistics, 2318–2335

2024

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

Daniil A Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes. 2023. Au- tonomous chemical research with large language models.Nature624, 7992 (2023), 570–578

2023

[2] [2]

Andres M Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller. 2024. ChemCrow: Augmenting large language models with chemistry tools.Nature Machine Intelligence6, 5 (2024), 525–535

work page 2024

[3] [3]

Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, Lilian Weng, and Aleksander Mądry. 2025. MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering. InThe Thirteenth International Conference on Learning Representations

work page 2025

[4] [4]

Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu

[5] [5]

Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, et al. 2024. AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors. In The Twelfth International Conference on Learning Representations

work page 2024

[6] [6]

Tenenbaum, and Igor Mor- datch

Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mor- datch. 2024. Improving Factuality and Reasoning in Language Models through Multiagent Debate. InProceedings of the 41st International Conference on Machine Learning. PMLR, 11733–11763

work page 2024

[7] [7]

Yonatan Geifman and Ran El-Yaniv. 2017. Selective Classification for Deep Neural Networks. InAdvances in Neural Information Processing Systems, Vol. 30

2017

[8] [8]

Antoine Grosnit, Alexandre Maraval, James Doran, Rasul Tutunov, Wenlong Wang, Haitham Bou Ammar, et al. 2024. Large Language Models Orchestrat- ing Structured Reasoning Achieve Kaggle Grandmaster Level.arXiv preprint arXiv:2411.03562(2024)

arXiv 2024