Pith. sign in

REVIEW 3 major objections 5 minor 46 references

An agent harness that records every hypothesis, test, evidence attachment, and belief change lets AI systems run scientific reasoning that humans and agents can actually audit.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 04:46 UTC pith:C6JD36EZ

load-bearing objection HEP is a practical harness that externalizes the H–T–E–B loop and shows planning agents do not spontaneously revise beliefs; solid systems paper, self-certified evidence is the known soft link. the 3 major comments →

arxiv 2607.09195 v1 pith:C6JD36EZ submitted 2026-07-10 cs.AI cond-mat.mtrl-sci

Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents

Izumi Takahara , Teruyasu Mizoguchi This is my paper
classification cs.AI cond-mat.mtrl-sci
keywords LLM agentsscientific discoveryhypothesis evolutionauditable AIbelief revisionmaterials scienceagent harnesshypothesis–test–evidence–belief cycle
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Current LLM agents can propose ideas and run tools, but their hypotheses, tests, and belief updates stay buried in free-text logs, so neither the agent nor a human can check whether evidence really moved the conclusion. This paper introduces the Hypothesis Evolution Protocol (HEP), a harness that turns each hypothesis into a persistent registry object with a belief probability that moves only when validated evidence is attached, plus explicit refine and merge operations and a lifecycle from proposal to supported, refuted, or dormant. On three open materials-science questions about which crystal structures a fixed potential prefers and why, a HEP-equipped agent ran the full hypothesis–test–evidence–belief cycle that a planning-style agent with the same model never externalized, reused the same protocol across tasks, and used more of the machinery as the base model grew more capable. A sympathetic reader cares because without an auditable record of belief revision, AI-driven discovery cannot be verified, contested, or safely built upon; HEP is offered as the missing footing for that record.

Core claim

A HEP-equipped agent operates the full hypothesis–test–evidence–belief cycle that planning-style agents leave unexercised, generalizes that disciplined cycle across materials research questions, and exploits the protocol more fully as base-LLM capability increases—constituting a step toward AI scientists whose scientific reasoning can be inspected, verified, and built upon.

What carries the argument

The Hypothesis Evolution Protocol (HEP): an append-only registry of persistent hypotheses (statement, belief P(H), lifecycle state, provenance) plus tools for propose/evolve, attach evidence, and transition state, so belief moves only on validated evidence and verdicts are registry-enforced thresholds.

Load-bearing premise

The agent’s own self-assigned belief numbers and self-certifications that a computational result is trustworthy are treated as a sufficient stand-in for real scientific belief revision.

What would settle it

Re-run the same materials tasks with HEP but replace the agent’s self-assessment of evidence validity with independent programmatic checks or a separate auditor agent; if the cycle collapses, hypotheses stay unresolved, or final beliefs no longer track externally validated outcomes, the claim that HEP yields auditable scientific reasoning fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Scientific reasoning of LLM agents becomes inspectable end-to-end rather than buried in unstructured logs.
  • The same harness can be reused across research questions without redesigning the scientific loop for each task.
  • Stronger base models automatically generate more hypotheses, deeper refine/merge trees, and fuller resolution of every hypothesis.
  • External validators can operate directly on the registry because every belief update is already a persistent, hash-chained record.
  • Harness and model capability are complementary: the harness externalizes reasoning; capability determines how fully it is exploited.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Until independent evidence validators are added, HEP may mainly audit the form of scientific reasoning rather than guarantee its soundness.
  • The same registry pattern could transfer to biology or chemistry automation wherever agents already have experimental or simulation tools.
  • Belief flips and parent–child lineage in the registry could become a practical metric for distinguishing discovery from mere search-and-rank.
  • As models improve, HEP-style harnesses may become the default interface that lets humans pick up, contest, or extend an AI run mid-stream.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes the Hypothesis Evolution Protocol (HEP), an agent harness that externalizes scientific hypothesis management as auditable operations. HEP maintains an append-only registry of persistent hypotheses, each with a natural-language statement, elicited belief P(H), lifecycle state (proposed, under_test, supported, refuted, dormant), and provenance, and exposes tools for proposal/evolution (de-novo, inspired-by, refine, merge), evidence attachment with self-certified validation, and state transitions gated by fixed belief thresholds (supported ≥ 0.8, refuted ≤ 0.2). On three open materials-science research questions about structure preferences of a fixed MLIP (AO2, A2BB'O6, MX), a HEP-equipped agent is shown to run the full hypothesis–test–evidence–belief cycle that a planning-style baseline largely omits (Fig. 2c), to generalize across tasks with terminal dispositions and evidence-driven belief flips (Fig. 3, Table 1), and to exploit the protocol more fully as base-LLM capability increases (Fig. 4, n=3).

Significance. If the results hold, HEP is a concrete, reusable scaffold that makes the scientific loop of LLM agents inspectable rather than buried in free-text logs—an important step toward verifiable AI scientists. Strengths include the explicit event-sourced registry, the planning baseline contrast with step-type fractions and transition matrices, cross-task generality, and the capability scaling study. The work is complementary to capability advances rather than a substitute for them, and the authors correctly flag independent auditors and higher-fidelity tools as next steps. The contribution is primarily methodological and systems-level rather than a new materials discovery per se.

major comments (3)
  1. [Methods; Results Fig. 2c, Fig. 3d] Methods (“Beliefs are elicited rather than computed”) and Results Fig. 2c / Fig. 3d: The central claim that HEP produces a genuine hypothesis–test–evidence–belief cycle rests on self-certified evidence validity—the agent alone states prior and updated P(H) and a trustworthiness diagnostic before belief is allowed to move. Attachments that fail self-certification leave belief unchanged; verdicts are then pure threshold transitions on those self-moved beliefs. The planning baseline shows that HEP elicits the H→T→E→B→J flow, but does not show that evidence actually constrains belief rather than that the agent produces protocol-compliant rationales. Given the paper’s own citation that prior agents ignore evidence in ~2/3 of traces, this is load-bearing for any reading of HEP as scientific belief revision rather than auditable bookkeeping. At minimum, the manuscript should (i) quantify how of
  2. [Results; Fig. 4] Results §“HEP scales with base-LLM capability” and Fig. 4: All quantitative claims (hypothesis counts, generation depths, mechanism composition, final-state fractions) rest on n=3 runs per condition. With that sample size, run-to-run instability (explicitly noted for GPT-4.1) makes the reported means and standard deviations fragile, and the capability-scaling gradient is only suggestive. Either increase n for the key comparisons or present the individual-run trajectories more prominently and soften the language of “scales with” to “trends with” pending larger samples.
  3. [Results; Table 1] Results Table 1 and accompanying text: The final “governing rules” are verified only by predicting preferred structures of held-out compositions under the same fixed MLIP, and the paper correctly notes that correspondence to experimental materials physics remains unexamined. For a claim of scientific discovery (even MLIP-level), the manuscript should either (a) report a quantitative held-out accuracy / confusion matrix for those predictions, or (b) more clearly label the endpoints as “MLIP-behavior summaries” rather than materials-physics rules, so that the contribution is not over-read as experimental discovery.
minor comments (5)
  1. [Fig. 2c] Fig. 2c: Rows H, J, and U for the planning-style agent rest on only 1–3 events; the caption notes this, but the main text should also caution against over-interpreting those transition probabilities.
  2. [Methods] Methods: The saturation stopping rule (no tool call for three consecutive iterations) and the 80-turn / 24-hour budgets are free parameters that can affect how thoroughly the cycle is run; a short sensitivity note or fixed-budget comparison would help.
  3. [Fig. 1; Methods] Fig. 1 and Methods: Clarify whether inspired-by edges ever affect lineage depth or only provenance; the text says provenance only, but the tree visualization in Fig. 2a still draws them, which can confuse readers.
  4. [Code Availability] Code availability is promised “upon publication”; for a protocol paper, a minimal public registry schema or tool interface at review time would strengthen reproducibility claims.
  5. [Abstract; Results] Minor wording: “planning-style agents lack” (abstract/results) is slightly absolute given that belief revision can be elicited by prompting; “did not spontaneously exhibit” matches the authors’ own later phrasing better.

Circularity Check

1 steps flagged

No load-bearing circular derivation: HEP evaluation is empirical against a planning baseline; only a minor self-definitional presentation of registry-enforced belief thresholds as if they were free findings.

specific steps
  1. self definitional [Results §HEP operates the cycle / Fig. 3d; Methods belief rules]
    "Verdicts are threshold transitions enforced by the registry: a hypothesis can be transitioned to supported only when P(H)≥0.8 and to refuted only when P(H)≤0.2... In the marginal histograms, the priors of eventually supported, refuted, and dormant hypotheses largely overlap, whereas the final beliefs separate cleanly by final state."

    Clean separation of final beliefs by terminal state is partly forced by the registry’s own thresholds: supported requires P≥0.8 and refuted P≤0.2. Presenting that separation as evidence that ‘verdicts were determined by accumulated evidence’ overstates an independent finding; the prior-overlap half of the argument is non-circular, but the final-belief half is definitional.

full rationale

This is a systems/protocol paper, not a first-principles derivation. HEP is defined independently of the materials answers; the central claims (full H–T–E–B–J cycle vs planning, generalization across AO2/A2BB′O6/MX, scaling with base LLM) rest on controlled runs with the same tools/goals and on run-length-independent step fractions and transition matrices. Those comparisons are not forced by definition: a planning-style agent with the same base LLM produces ~0% belief-update steps and never externalizes B, while HEP runs show E→B and B→J flow. Self-citation is present only as infrastructure (e.g. AtomisticSkills, prior materials agents) and is not load-bearing for uniqueness or for the cycle claim. The skeptic’s point that beliefs and evidence validity are self-elicited/self-certified is a real correctness/validity risk—the paper itself flags independent auditors as future work—but it is not circularity under the analyzer’s criteria: it does not make a claimed prediction equal its fitted input by construction, nor does it smuggle a uniqueness theorem via self-citation. The only mild circularity is presentational: final-belief separation by lifecycle state is partly enforced by the registry thresholds (supported only if P(H)≥0.8, refuted only if P(H)≤0.2), so that particular histogram pattern is not an independent empirical discovery. That does not collapse the comparative results. Score 2.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 3 invented entities

The central claims rest on a designed protocol plus empirical agent runs, not on fitted physical constants. Load-bearing design choices are the belief thresholds, self-elicited beliefs, evidence validation gate, and the materials task framing with a fixed MLIP. No new physical entities are postulated; the invented entities are software constructs of the harness.

free parameters (5)
  • support_threshold P(H) >= 0.8
    Registry-enforced threshold for transitioning a hypothesis to supported; chosen by design, not derived from data.
  • refutation_threshold P(H) <= 0.2
    Registry-enforced threshold for transitioning a hypothesis to refuted; design choice that shapes all verdicts.
  • agent-elicited prior and updated P(H)
    Belief numbers are stated by the LLM with rationales rather than computed from a calibrated model; every trajectory depends on these self-assigned values.
  • saturation stopping rule (no tool call for 3 consecutive iterations)
    Run termination criterion that determines when the hypothesis population is considered complete.
  • outer-loop turn budget (up to 80 LLM turns per iteration) and 24-hour wall-clock budget
    Operational limits that constrain how deeply the agent can explore and evolve hypotheses.
axioms (5)
  • domain assumption Scientific reasoning by LLM agents can be usefully externalized as discrete hypothesis objects with numeric beliefs updated only by attached evidence.
    Core design premise of HEP (Introduction and Results); not proved, but the evaluation tests whether agents will operate under it.
  • domain assumption A planning-style plan-execute-replan agent is a fair baseline for whether the hypothesis-test-evidence-belief cycle emerges without HEP.
    Used for the Fig. 2c comparison; alternative scaffolds (belief-oriented prompting, other agent frameworks) are not ablated.
  • domain assumption MLIP-level energetics and structural analyses are adequate tests for the materials research questions posed.
    Stated limitation in Results/Conclusions: evidence is MLIP-level, not first-principles or experimental, and correspondence to established materials physics is left open.
  • ad hoc to paper Self-certification that computational/analytical evidence is trustworthy is sufficient for a belief update to count as validated.
    Methods: belief moves only if the agent certifies setup adequacy and result validity; uncertified attachments leave belief unchanged.
  • domain assumption Standard LLM tool-use and agent-loop machinery (OpenAI Agents SDK, MCP tools, AtomisticSkills) correctly execute the intended scientific actions.
    Agent setup in Methods; failures would confound both HEP and baseline runs.
invented entities (3)
  • HEP Registry (append-only, event-sourced hypothesis store) no independent evidence
    purpose: Persist each hypothesis with belief, lifecycle state, provenance, and auditable event log.
    Software construct introduced by the paper; independent_evidence is false because it is defined by the protocol itself.
  • Hypothesis lifecycle states (proposed, under_test, supported, refuted, dormant) no independent evidence
    purpose: Force every hypothesis toward an explicit terminal disposition under registry rules.
    Protocol invention; transitions are tool-mediated and threshold-gated.
  • Four generation mechanisms (de-novo, inspired-by, refine, merge) no independent evidence
    purpose: Structure how new hypotheses enter and evolve into an explicit lineage.
    Design taxonomy of the harness; refine/merge accepted only for parents with evidence or verdicts.

pith-pipeline@v1.1.0-grok45 · 16738 in / 3494 out tokens · 33529 ms · 2026-07-13T04:46:25.958508+00:00 · methodology

0 comments
read the original abstract

Large language model (LLM) agents are increasingly expected to play a central role in AI-driven scientific discovery. Equipped with broad knowledge, flexible reasoning, and tool use, they have the potential to autonomously explore and solve scientific problems by repeatedly proposing hypotheses, testing them, and revising their beliefs in the light of the evidence. In current agents, however, these hypotheses, tests, and belief updates are buried in unstructured logs, and no mechanism lets the agent or the human researcher audit that process. Here we propose the Hypothesis Evolution Protocol (HEP), an agent harness that provides hypothesis generation, evaluation, and evolution as explicit, auditable operations. On materials-science research tasks, a HEP-equipped agent operates the hypothesis--test--evidence--belief cycle that planning-style agents lack, generalizes across research questions, and exploits the protocol more fully as the base LLM becomes more capable. These results mark a step toward auditable AI scientists, whose scientific reasoning can be inspected, verified, and built upon.

Figures

Figures reproduced from arXiv: 2607.09195 by Izumi Takahara, Teruyasu Mizoguchi.

Figure 1
Figure 1. Figure 1: Overview of the Hypothesis Evolution Protocol (HEP). (a) [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: HEP makes the scientific loop explicit and auditable. (a) [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Behavior of the HEP-equipped agent across different research questions. (a) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: LLM dependence of the ability to operate HEP. (a) [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 15 linked inside Pith

  1. [1]

    Towardsend-to-endautomationofairesearch.Nature,651(8107):914–919, 2026

    ChrisLu,CongLu,RobertTjarkoLange,YutaroYamada,ShengranHu,JakobFoerster,David Ha,andJeffClune. Towardsend-to-endautomationofairesearch.Nature,651(8107):914–919, 2026

    2026

  2. [2]

    Accelerating scientific discovery with co-scientist.Nature, 2026

    Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Petar Sirkovic, Artiom Myaskovsky, Grzegorz Glowaty, Felix Weissenberger, Alessio Orlandi, Dan Popovici, et al. Accelerating scientific discovery with co-scientist.Nature, 2026. Published online 19 May 2026; preprint: arXiv:2502.18864

    Pith/arXiv arXiv 2026

  3. [3]

    Landsness, Daniel L

    LudovicoMitchener,AngelaYiu,BenjaminChang,MathieuBourdenx,TylerNadolski,Arvis Sulovari, Eric C. Landsness, Daniel L. Barabasi, Siddharth Narayanan, Nicky Evans, et al. Kosmos: An ai scientist for autonomous discovery.arXiv preprint arXiv:2511.02824, 2025. 14

    Pith/arXiv arXiv 2025

  4. [4]

    Towards agentic intelligence for materials science

    Huan Zhang, Yizhan Li, Wenhao Huang, Ziyu Hou, Yu Song, Xuye Liu, Farshid Effaty, Jinya Jiang, Sifan Wu, Qianggang Ding, et al. Towards agentic intelligence for materials science. arXiv preprint arXiv:2602.00169, 2026

    arXiv 2026

  5. [5]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. InInternational Conference on Learning Representations (ICLR), 2023

    2023

  6. [6]

    Introducing the Model Context Protocol.https://www.anthropic.com/news/ model-context-protocol, 2024

    Anthropic. Introducing the Model Context Protocol.https://www.anthropic.com/news/ model-context-protocol, 2024. Specification:https://modelcontextprotocol.io; ac- cessed 2026-06-23

    2024

  7. [7]

    Model context protocol (mcp): Landscape, security threats, and future research directions.arXiv preprint arXiv:2503.23278, 2025

    Xinyi Hou, Yanjie Zhao, Shenao Wang, and Haoyu Wang. Model context protocol (mcp): Landscape, security threats, and future research directions.arXiv preprint arXiv:2503.23278, 2025

    Pith/arXiv arXiv 2025

  8. [8]

    Voyager: An open-ended embodied agent with large language models.Transactions on Machine Learning Research (TMLR), 2024

    GuanzhiWang,YuqiXie,YunfanJiang,AjayMandlekar,ChaoweiXiao,YukeZhu,LinxiFan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models.Transactions on Machine Learning Research (TMLR), 2024

    2024

  9. [9]

    Sok: Agentic skills – beyond tool use in llm agents.arXiv preprint arXiv:2602.20867, 2026

    Yanna Jiang, Delong Li, Haiyu Deng, Baihe Ma, Xu Wang, Qin Wang, and Guangsheng Yu. Sok: Agentic skills – beyond tool use in llm agents.arXiv preprint arXiv:2602.20867, 2026

    Pith/arXiv arXiv 2026

  10. [10]

    Mem0: Building production-ready ai agents with scalable long-term memory.arXiv preprint arXiv:2504.19413, 2025

    Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready ai agents with scalable long-term memory.arXiv preprint arXiv:2504.19413, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    A survey of self-evolving agents: What, when, how, and where to evolve on the path to artificial super intelligence.Transactions on Machine Learning Research, 2026

    Huan-ang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi, Qihan Ren, et al. A survey of self-evolving agents: What, when, how, and where to evolve on the path to artificial super intelligence.Transactions on Machine Learning Research, 2026. Survey Certification

    2026

  12. [12]

    Adaplanner: Adaptive planning from feedback with language models

    HaotianSun,YuchenZhuang,LingkaiKong,BoDai,andChaoZhang. Adaplanner: Adaptive planning from feedback with language models. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  13. [13]

    Reflexion: Language agents with verbal reinforcement learning

    NoahShinn,FedericoCassano,EdwardBerman,AshwinGopinath,KarthikNarasimhan,and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  14. [14]

    Toward generalist autonomous research via hypothesis- tree refinement.arXiv preprint arXiv:2606.11926, 2026

    Jiajie Jin, Yuyang Hu, Kai Qiu, Qi Dai, Chong Luo, Guanting Dong, Xiaoxi Li, Tong Zhao, Xiaolong Ma, Gongrui Zhang, et al. Toward generalist autonomous research via hypothesis- tree refinement.arXiv preprint arXiv:2606.11926, 2026. 15

    Pith/arXiv arXiv 2026

  15. [15]

    Szostkiewicz, Jon M

    Ali Essam Ghareeb, Benjamin Chang, Ludovico Mitchener, Angela Yiu, Caralyn J. Szostkiewicz, Jon M. Laurent, Muhammed T. Razzak, Andrew D. White, Michaela M. Hinks, and Samuel G. Rodriques. Robin: A multi-agent system for automating scientific discovery. arXiv preprint arXiv:2505.13400, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    Biodisco: Multi-agent hypothesis generation with dual-mode evidence, iterative feedback and temporal evaluation.arXiv preprint arXiv:2508.01285, 2025

    Yujing Ke, Kevin George, Kathan Pandya, David Blumenthal, Maximilian Sprang, Gerrit Großmann, Sebastian Vollmer, and David Antony Selby. Biodisco: Multi-agent hypothesis generation with dual-mode evidence, iterative feedback and temporal evaluation.arXiv preprint arXiv:2508.01285, 2025

    arXiv 2025

  17. [17]

    Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D

    Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D. White, and Philippe Schwaller. Augmenting large language models with chemistry tools.Nature Machine Intelli- gence, 6(5):525–535, 2024

    2024

  18. [18]

    Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes

    Daniil A. Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes. Autonomous chemical research with large language models.Nature, 624(7992):570–578, 2023

    2023

  19. [19]

    Organa: A robotic assistant for automated chemistry experimentation and characterization.Matter, 8(2):101897, 2025

    Kourosh Darvish, Marta Skreta, Yuchi Zhao, Naruki Yoshikawa, Sagnik Som, Miroslav Bog- danovic, Yang Cao, Han Hao, Haoping Xu, Alán Aspuru-Guzik, et al. Organa: A robotic assistant for automated chemistry experimentation and characterization.Matter, 8(2):101897, 2025

    2025

  20. [20]

    Llmatdesign: Autonomousmaterialsdiscoverywith large language models.arXiv preprint arXiv:2406.13163, 2024

    ShuyiJia,ChaoZhang,andVictorFung. Llmatdesign: Autonomousmaterialsdiscoverywith large language models.arXiv preprint arXiv:2406.13163, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Accelerated inorganic materials design with generative ai agents.Cell Reports Physical Science, 6(12), 2025

    Izumi Takahara, Teruyasu Mizoguchi, and Bang Liu. Accelerated inorganic materials design with generative ai agents.Cell Reports Physical Science, 6(12), 2025

    2025

  22. [22]

    Crystalyse: a multi-tool agent for materials design.arXiv preprint arXiv:2512.00977, 2025

    Ryan Nduma, Hyunsoo Park, and Aron Walsh. Crystalyse: a multi-tool agent for materials design.arXiv preprint arXiv:2512.00977, 2025

    arXiv 2025

  23. [23]

    Materealize: a multi-agent deliberation system for end-to-end material design and synthesis.arXiv preprint arXiv:2601.15743, 2026

    SeongminKim,JaehwanChoi,KunikJang,JunkilPark,VariniaBernales,AlánAspuru-Guzik, and Yousung Jung. Materealize: a multi-agent deliberation system for end-to-end material design and synthesis.arXiv preprint arXiv:2601.15743, 2026

    arXiv 2026

  24. [24]

    El agente: An autonomous agent for quantum chemistry.Matter, 8(7):102263, 2025

    YunhengZou,AustinH.Cheng,AbdulrahmanAldossary,JiaruBai,ShiXuanLeong,JorgeAr- turoCampos-Gonzalez-Angulo,ChanghyeokChoi,CherTianSer,GaryTom,AndrewWang, et al. El agente: An autonomous agent for quantum chemistry.Matter, 8(7):102263, 2025

    2025

  25. [25]

    Pham, Aditya Tanikanti, and Murat Keçeli

    Thang D. Pham, Aditya Tanikanti, and Murat Keçeli. Chemgraph as an agentic framework for computational chemistry workflows.Communications Chemistry, 9(1):33, 2026

    2026

  26. [26]

    Harnessing atomisticskills for agentic atomistic research.arXiv preprint arXiv:2605.24002, 2026

    Bowen Deng, Bohan Li, Matthew Cox, Hoje Chun, Juno Nam, Artur Lyssenko, Sathya Edamadaka, Jurgis Ruza, Xiaochen Du, Nofit Segal, et al. Harnessing atomisticskills for agentic atomistic research.arXiv preprint arXiv:2605.24002, 2026. 16

    Pith/arXiv arXiv 2026

  27. [27]

    Agenticllmreasoninginaself-driving laboratory for air-sensitive lithium halide spinel conductors.arXiv preprint arXiv:2604.11957, 2026

    Yuxing Fei, Bernardus Rendy, Xiaochen Yang, Junhee Woo, Xu Huang, Chang Li, Shilong Wang,DavidMilsted,YanZeng,andGerbrandCeder. Agenticllmreasoninginaself-driving laboratory for air-sensitive lithium halide spinel conductors.arXiv preprint arXiv:2604.11957, 2026

    Pith/arXiv arXiv 2026

  28. [28]

    Performance of ai agents based on reasoning language models on ald process optimization tasks.arXiv preprint arXiv:2601.09980, 2026

    Angel Yanguas-Gil. Performance of ai agents based on reasoning language models on ald process optimization tasks.arXiv preprint arXiv:2601.09980, 2026

    arXiv 2026

  29. [29]

    Knowledge-driven autonomous materials research via collaborative multi-agent and robotic system.Matter, 9(2):102577, 2026

    Tongyu Shi, Yutang Li, Zhanlong Wang, Wenhe Xu, Guolai Jiang, Dawei Dai, Jie Zhou, Hao Huang, Rui He, Seeram Ramakrishna, et al. Knowledge-driven autonomous materials research via collaborative multi-agent and robotic system.Matter, 9(2):102577, 2026

    2026

  30. [30]

    Smedskjaer, Katrin Wondraczek, Lothar Wondraczek, Nitya Nand Gosvami, and N

    Indrajeet Mandal, Jitendra Soni, Mohd Zaki, Morten M. Smedskjaer, Katrin Wondraczek, Lothar Wondraczek, Nitya Nand Gosvami, and N. M. Anoop Krishnan. Evaluating large language model agents for automation of atomic force microscopy.Nature Communications, 16:9104, 2025

    2025

  31. [31]

    Prince, Tao Zhou, Henry Chan, and Mathew J

    Aikaterini Vriza, Michael H. Prince, Tao Zhou, Henry Chan, and Mathew J. Cherukara. Op- eratingadvancedscientificinstrumentswithaiagentsthatlearnonthejob.npjComputational Materials, 12:160, 2026

    2026

  32. [32]

    Aiscientistsproduce results without reasoning scientifically.arXiv preprint arXiv:2604.18805, 2026

    Martiño Ríos-García, Nawaf Alampara, Chandan Gupta, Indrajeet Mandal, Sajid Mannan, AliAsgharAghajani,N.M.AnoopKrishnan,andKevinMaikJablonka. Aiscientistsproduce results without reasoning scientifically.arXiv preprint arXiv:2604.18805, 2026

    Pith/arXiv arXiv 2026

  33. [33]

    Alireza Ghafarollahi and Markus J. Buehler. Sciagents: Automating scientific discov- ery through bioinspired multi-agent intelligent graph reasoning.Advanced Materials, 37(22):2413523, 2025

    2025

  34. [34]

    Large language models for automated open-domain scientific hypotheses discovery

    Zonglin Yang, Xinya Du, Junxian Li, Jie Zheng, Soujanya Poria, and Erik Cambria. Large language models for automated open-domain scientific hypotheses discovery. InFindings of the Association for Computational Linguistics (ACL Findings), pages 13545–13565, 2024

    2024

  35. [35]

    Moose-chem: Large language models for rediscovering unseen chemistry scientific hypotheses

    Zonglin Yang, Wanhao Liu, Ben Gao, Tong Xie, Yuqiang Li, Wanli Ouyang, Soujanya Poria, Erik Cambria, and Dongzhan Zhou. Moose-chem: Large language models for rediscovering unseen chemistry scientific hypotheses. InThe Thirteenth International Conference on Learning Representations (ICLR), 2025

    2025

  36. [36]

    Moose-chem2: Exploringllmlimitsinfine-grainedscien- tific hypothesis discovery via hierarchical search

    ZonglinYang,WanhaoLiu,BenGao,YujieLiu,WeiLi,TongXie,LidongBing,WanliOuyang, ErikCambria,andDongzhanZhou. Moose-chem2: Exploringllmlimitsinfine-grainedscien- tific hypothesis discovery via hierarchical search. InAdvances in Neural Information Processing Systems (NeurIPS), 2025. 17

    2025

  37. [37]

    Aigs: Generating science from ai-powered automated falsification.arXiv preprint arXiv:2411.11910, 2024

    Zijun Liu, Kaiming Liu, Yiqi Zhu, Xuanyu Lei, Zonghan Yang, Zhenhe Zhang, Peng Li, and Yang Liu. Aigs: Generating science from ai-powered automated falsification.arXiv preprint arXiv:2411.11910, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    Li, Emmanuel Candès, and Jure Leskovec

    Kexin Huang, Ying Jin, Ryan Li, Michael Y. Li, Emmanuel Candès, and Jure Leskovec. Auto- mated hypothesis validation with agentic sequential falsifications. InProceedings of the 42nd International Conference on Machine Learning (ICML), 2025

    2025

  39. [39]

    Autodiscovery: Open-ended scientific discovery via bayesian surprise

    Dhruv Agarwal, Bodhisattwa Prasad Majumder, Reece Adamson, Megha Chakravorty, Satvika Reddy Gavireddy, Aditya Parashar, Harshit Surana, Bhavana Dalvi Mishra, Andrew McCallum, Ashish Sabharwal, et al. Autodiscovery: Open-ended scientific discovery via bayesian surprise. InAdvances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  40. [40]

    Wang, Lee Marom, Subhadeep Pal, Rachel K

    Fiona Y. Wang, Lee Marom, Subhadeep Pal, Rachel K. Luu, Wei Lu, Jaime A. Berkovich, and MarkusJ.Buehler. Autonomousagentscoordinatingdistributeddiscoverythroughemergent artifact exchange.arXiv preprint arXiv:2603.14312, 2026

    arXiv 2026

  41. [41]

    Discoverphysics: Benchmarking llms for out-of-the-box scientific thinking.arXiv preprint arXiv:2605.26087, 2026

    MattL.Wiemann,LindsayM.Smith,PeterMelchior,SiddharthMishra-Sharma,AndrewGor- don Wilson, Pavel Izmailov, and Carolina Cuesta-Lázaro. Discoverphysics: Benchmarking llms for out-of-the-box scientific thinking.arXiv preprint arXiv:2605.26087, 2026

    Pith/arXiv arXiv 2026

  42. [42]

    Workflow closure is not scientific closure in auto-research systems.arXiv preprint arXiv:2605.26200, 2026

    Shuai Wang, Xinyuan Tian, Pangpang Liu, and Yize Zhao. Workflow closure is not scientific closure in auto-research systems.arXiv preprint arXiv:2605.26200, 2026

    Pith/arXiv arXiv 2026

  43. [43]

    Harshit Bisht, Vinay Kumar, Kevin Maik Jablonka, Mausam, and N. M. Anoop Krish- nan. Agentic ai scientists are not built for autonomous scientific discovery.arXiv preprint arXiv:2605.08956, 2026

    Pith/arXiv arXiv 2026

  44. [44]

    GPT-5.5 System Card.https://openai.com/index/gpt-5-5-system-card/, 2026

    OpenAI. GPT-5.5 System Card.https://openai.com/index/gpt-5-5-system-card/, 2026. Published 2026-04-23; accessed 2026-07-09

    2026

  45. [45]

    Introducing GPT-5.4 mini and nano.https://openai.com/index/ introducing-gpt-5-4-mini-and-nano/, 2026

    OpenAI. Introducing GPT-5.4 mini and nano.https://openai.com/index/ introducing-gpt-5-4-mini-and-nano/, 2026. Published 2026-03-17; accessed 2026-07-09

    2026

  46. [46]

    Introducing GPT-4.1 in the API.https://openai.com/index/gpt-4-1/, 2025

    OpenAI. Introducing GPT-4.1 in the API.https://openai.com/index/gpt-4-1/, 2025. Accessed 2026-07-09

    2025