Pith. sign in

REVIEW 4 major objections 5 minor 7 cited by

InternAgent: When Agent Becomes the Scientist -- Building Closed-Loop System from Hypothesis to Verification

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read InternAgent closes the loop from hypothesis to verification, improving baseline performance on all 12 scientific and AI tasks.

desk verdict Real engineering, unreliable numbers: the framework is a solid systems contribution but the headline gains are best-of-N artifacts rather than demonstrated idea quality. read the letter →

arxiv 2505.16938 v3 pith:24TDR375 submitted 2025-05-22 cs.AI cs.CLcs.CV

InternAgent Team: Bo Zhang , Shiyang Feng , Xiangchao Yan , Jiakang Yuan , Runmin Ma , Yusong Hu , Zhiyin Yu , Xiaohan He
show 17 more authors
Songtao Huang Shaowei Hou Zheng Nie Zhilong Wang Jinyao Liu Tianshuo Peng Peng Ye Dongzhan Zhou Shufei Zhang Xiaosong Wang Yilan Zhang Meng Li Zhongying Tu Xiangyu Yue Wangli Ouyang Bowen Zhou Lei Bai
This is my paper · ORCID
classification cs.AIcs.CLcs.CV
keywords multi-agentsystemsautonomousscientificresearchlargelanguagemodelshypothesisgenerationidea-to-methodologyautomatedcodeanddebuggingclosed-looppipelineAIforscience
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

InternAgent is an LLM-based multi-agent system that claims to automate the entire research loop: survey literature, generate and evolve candidate ideas, turn an idea into a step-by-step methodology, implement it as code, debug it, run experiments, and feed results back for another round. The paper's central evidence is that across 12 tasks—from chemical reaction-yield prediction and enhancer-activity prediction to semantic segmentation and vision-language fine-tuning—InternAgent improved the given baseline's score on every task, with headline gains such as reaction-yield $R^2$ rising from 27.6 to 35.4 in 12 hours and enhancer-activity correlation rising from 0.65 to 0.79 in 4 hours. A sympathetic reader would care because, if this holds, it shifts the bottleneck of applied research from manual implementation and trial-and-error to articulating the task; the framework would be a reusable instrument for turning hypotheses into verified code. The paper also claims that human experts rate InternAgent-generated ideas higher in soundness, contribution, and overall quality than ideas from prior automated research systems, and that the monetary cost per idea is only a few dollars.

What carries the argument

The central mechanism is the closed-loop research cycle, carried by four connected modules. First, self-evolving idea generation: a Survey Agent reads roughly 50 papers, an Idea Innovation Agent generates 15 initial ideas and repeatedly evolves each into 3 variants, an Assessment Agent scores ideas on coherence, credibility, verifiability, novelty, and alignment while preserving diversity, and human feedback can steer the process. Second, idea-to-methodology construction: a Method Development Agent expands an idea into a formal, step-by-step method description, including pseudocode and equations, before coding starts. Third, evolutionary experimental planning and execution: the coder, Aider for small files and OpenHands for repositories, implements the method, and an exception-guided debugger fixes failures, with up to four debug attempts. Fourth, adaptive evolution: the Orchestration Agent re-plans after each run, keeping only components that empirically improve performance. This sequence is what lets an idea become a verified experimental result rather than just a proposal.

What would settle it

Run InternAgent on the same 12 tasks with the idea-generation and self-evolution stages replaced by a random or fixed set of no-op 'ideas' while keeping the same code-debug and rerun budget; if the no-op pipeline matches InternAgent's improvements, the closed-loop hypothesis generation is not what drives the results.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that a single closed-loop multi-agent architecture can carry a research project from a natural-language task description to working improved code with no human in the loop except optional feedback. The architecture's load-bearing move is to convert each high-level idea into a detailed methodology, with formal equations and step-by-step algorithm descriptions, before any code is written; the paper argues this is why the coder can actually implement the idea. Multi-round experimental planning then rolls the methodology's components into the baseline incrementally, adapting the plan after each run. The quantitative claim is that InternAgent outperforms the baseline on all 12 tasks and outperforms DOLPHIN on every task where DOLPHIN runs, with the best ideas also more stable across repeated runs in the few-shot yield-prediction study. In human evaluation, reviewers gave InternAgent's ideas higher soundness, contribution, and overall ratings than AI-Scientist-V2's ideas across the four tasks compared.

Load-bearing premise

The load-bearing premise is that the baselines are strong, honestly reproduced reference points and that the gains come from InternAgent's generated ideas and methodology rather than from its extra search and rerun budget, which includes 10 candidate ideas, idea evolution, up to 4 debug attempts, and adaptive re-running of experiments.

Editorial extensions

If this is right

  • If correct, an autonomous research pipeline can take a task description and baseline repository and return improved, working code within hours, at API costs of roughly $0.6 to $1.2 per idea and training costs of 0.1 to 192 A100 GPU-hours per task.
  • The claimed gains across chemistry, molecular dynamics, power flow, time series, genomics, NLP, 2D and 3D vision, and VLM fine-tuning indicate the method is domain-agnostic rather than tuned to one benchmark.
  • Because the framework handles repo-level codebases in Auto2DSeg, AutoPCDet, and AutoVLM, it extends automated research beyond single-file toy experiments.
  • Human experts rated the generated ideas higher on soundness, contribution, and overall quality than those from prior automated systems, suggesting the pipeline produces proposals experts find plausible, not just code that runs.
  • The adaptive-evolution ablation in Table 8 indicates that iterative plan-refinement after each run is responsible for part of the gain; without it, success rates and improvements drop.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The reported numbers do not separate idea quality from search budget; a budget-matched comparison against random ideas or a hyperparameter search would clarify how much of the gain is the 'scientist' versus the compute spent.
  • Editorial inference: The idea-to-methodology step, with formal algorithms and equations, could be reused as a benchmark itself, allowing tests of which parts of a generated method description are necessary for a coder to reproduce the intended change.
  • Editorial inference: The same closed loop could be pointed at open problems where the 'experiment' is simulation or data analysis rather than training runs, such as choosing priors, designing loss functions, or selecting ablation studies, as long as evaluation is scriptable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces InternAgent, a closed-loop multi-agent framework that autonomously performs scientific research: it generates and evolves research ideas, converts them into detailed methodologies, implements them in code, runs experiments, and iterates. The authors report experiments across 12 diverse tasks (reaction yield prediction, molecular dynamics, power flow estimation, time series forecasting, transcription prediction, enhancer activity prediction, sentiment classification, image classification, 3D point cloud classification, semantic segmentation, autonomous driving, and VLM fine-tuning). The central claim is that InternAgent's generated ideas improve baseline code across all 12 tasks, with headline examples including AutoRYP R2 from 27.6 to 35.4, AutoEAP HK-PCC from 0.65 to 0.79, and Auto2DSeg mIoU from 78.8 to 81.0. The paper also compares against DOLPHIN and AI-Researcher and presents a human evaluation of idea novelty.

Significance. If the central claim were established with controlled comparisons, this would be an important demonstration of end-to-end LLM-agent research: the task coverage is unusually broad, the full pipeline from idea generation to experimental verification is explicit, and the authors have open-sourced both baselines and generated code, which is a real reproducibility asset. However, the stress-test concern lands squarely: the experimental protocol does not isolate the contribution of generated ideas from the large search and retry budget, and several headline numbers are selected extrema or censored averages. The paper contains no formal derivation and no fitted quantitative model, so the issue is not circularity but attribution: the evidence as presented cannot support the abstract's claim that InternAgent's ideas, rather than its search budget, are responsible for the reported gains.

major comments (4)
  1. [Sec. 3.1.3; Tables 1, 2, 8] The central attribution claim is not supported by the experimental protocol. InternAgent is given a large budget: 10 tested ideas per task, up to 4 debug attempts, and up to 5 adaptive reruns (Aider) or 3 (OpenHands), while the baselines and DOLPHIN are not given the same number of random restarts, hyperparameter trials, or debug iterations. Table 8 shows that the adaptive-evolution rerun loop alone changes Auto2DCls max accuracy from 81.6 to 83.3 and AutoRYP improved/successful from 2/5 to 4/6, so a substantial part of the reported gain is the retry/search mechanism rather than the content of the generated ideas. Without a baseline-equivalent search control (for example, best-of-N baseline runs, or InternAgent without adaptive evolution and with one run per idea), the abstract's claim that InternAgent 'generates innovative ideas to enhance baseline code' is not established.
  2. [Sec. 3.2; Tables 1-4] The 'Average Performance' reported in Tables 1 and 2 is defined as the average across experiments with performance gains, which is a censored statistic. Tables 3 and 4 show that many ideas fail to run or fail to improve: AutoVLM has only 1 improving idea out of 5 successful runs (1/5/10), so its reported average of 67.6 is simply the single success, and AutoPCDet has 2/5/10. Reporting only the censored average makes the 'versatility across 12 tasks' claim appear stronger than the data warrant. The paper should report the full distribution over all 10 ideas, including failed runs, using an uncensored mean or median, and should present per-idea results where feasible.
  3. [Table 7 vs. Table 1 and Abstract] The baseline number for AutoRYP is inconsistent across claims. Table 1 and the abstract use baseline R2 = 27.6, but Section 1 and Table 7 report the baseline as 24.2 ± 4.2 over five repeated runs, and 27.6 is the maximum of those runs. The headline '27.6 to 35.4' therefore compares a best-of-N baseline against a best-of-N-plus-retries agent, while the mean-to-mean comparison in Table 7 is 24.2 to 34.8. The paper must state which statistic is being compared and use the same protocol for both the baseline and the agent; otherwise the reported improvement is inflated by seed selection.
  4. [Table 10 and Appendix B] The human evaluation of idea novelty does not currently provide credible evidence for the novelty claim. The paper does not report whether evaluators were blinded to the source of the ideas, whether they were independent of the project team (Appendix A lists an 'Idea Human Evaluation Specialist' group within the project team), or any inter-rater reliability statistic. Appendix B.1 promises per-idea scores but only provides scoring criteria, not the detailed scores. Without independent, blinded review and agreement measures, the large score gaps in Table 10 could reflect expectation bias, and the claim that InternAgent generates more novel ideas than AI-Scientist-V2 is not substantiated.
minor comments (5)
  1. [Sec. 3.1.3; Tables 1-4] Section 3.1.3 says the idea generation agent generates 15 ideas and selects the top 5 after evolution, while Tables 1-4 state that 10 ideas are tested per task; please clarify how the 10 tested ideas are produced and how they relate to the 15 initial ideas and the evolution tree.
  2. [Abstract; Sec. 3.1.2] The abstract says 'precision advanced' for 2D semantic segmentation, but the metric defined in Sec. 3.1.2 and reported in Table 2 is mIoU, not precision; please use the correct metric name.
  3. [Tables 5, 6, and 9] The cost tables report training time and per-idea generation and debugging costs, but they do not report the total number of runs or the cost of the adaptive-evolution reruns; the 'Total Cost' in Table 9 should be defined consistently with the per-run costs in Tables 5 and 6.
  4. [Table 9] In Table 9, AI-Scientist-V2 is listed without performance numbers, and the text states it struggled to write runnable code; please provide the observed success rates, number of executable runs, and any partial results so the comparison is transparent.
  5. [Sec. 1] The statement that human researchers typically require several months to achieve a similar level of performance improvement is unsupported; please either provide a citation or soften the claim.

Circularity Check

1 steps flagged · score 2.0 of 10

No derivation-level circularity: InternAgent's headline gains are direct measurements; the one definitional issue is the success-conditioned 'Average Performance' statistic.

  1. self definitional [Sec. 3.2, Experimental Results (definition of 'Average Performance' for Tables 1 and 2)]
    "Tab. 1 and Tab. 2 show the max performance and average performance (i.e., the average performance across experiments with performance gains) achieved by InternAgent and DOLPHIN."

    The 'Average Performance' column is defined as the mean of only the runs that already improved over baseline, so every reported average improvement is entailed by the selection rule rather than by the quality of a typical generated idea. Once the set of gaining runs is nonempty, the average of that set is above baseline by construction. Table 4 makes the reduction concrete: AutoVLM has 1 improving / 5 successful / 10 tested ideas, and Table 2 reports the same 67.6 as both Max and Average, i.e., the average is just the single successful run while the four non-improving runs are discarded. The column therefore restates the success condition instead of measuring expected InternAgent performance.

full rationale

The paper contains no derivation chain in which an output quantity is constructed from its own inputs; the 12-task claims are empirical test-set measurements of code variants produced by the agent pipeline. The headline max-performance gains (e.g., AutoRYP 27.6 to 35.4, AutoEAP 0.65 to 0.79, Auto2DSeg 78.8 to 81.0) are best-of-N selections over 10 generated ideas, which is a search-budget and selection issue rather than circularity: an existence claim about capability can legitimately be supported by a maximum. The one genuine by-construction relationship is the 'Average Performance' statistic, which is defined conditional on gains and therefore cannot fail to show improvement once at least one idea succeeds; this is flagged as a self_definitional step. The DOLPHIN comparison and the human novelty evaluation involve same-group or author-internal assessors, but those are independence and robustness concerns, not load-bearing self-citations that reduce the argument to itself. Overall, the central empirical claim retains independent content, so the circularity score is low.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim is empirical, so the main axioms are assumptions about experimental fairness, LLM reliability, and evaluator independence. The paper does not introduce new mathematical entities; its hand-set procedural budgets are listed as free parameters because the headline results depend on them and no sensitivity analysis is provided.

free parameters (3)
  • Idea generation and evolution budget = 15 initial ideas, each evolved into 3 children, up to 4 evolution rounds, top 5 selected
    These hand-set budgets determine how many ideas are tried and which ones reach the coding stage. The paper provides no sensitivity analysis over these values, and the headline max results depend on trying many ideas.
  • Coder retry budgets = max debug attempts = 4; max runs = 5 for Aider, 3 for OpenHands
    The success rate and final performance depend on how many debug and rerun attempts are allowed. These are chosen by hand and not ablated except for the adaptive evolution component on three tasks.
  • Survey agent paper count = 50 papers reviewed
    The survey agent reads 50 papers to ground idea generation. This number is fixed by the authors and no analysis shows how performance varies with survey depth.
assumptions (3)
  • domain assumption Baseline implementations are strong, fairly tuned reference points.
    The paper reports baseline numbers as run locally, but does not compare against published best-known numbers for each baseline or provide repeated-seed evidence. If baselines are undertuned, the reported gains overstate InternAgent's value. This assumption enters in Tables 1 and 2 and in the comparison protocol of Sec. 3.2.
  • domain assumption The LLM services used inside InternAgent (GPT-4o, Claude-3.7-Sonnet) behave reliably enough for idea generation, methodology writing, and debugging.
    The entire pipeline depends on the undocumented behavior of commercial LLM APIs. The paper reports execution success rates, but does not control for prompt sensitivity, model drift, or version changes. This assumption is central to the claim that gains come from the framework rather than from the specific LLM.
  • domain assumption Human reviewers provide unbiased, reliable novelty scores.
    Appendix B describes reviewer qualifications and the scoring rubric, but does not report reviewer independence from the authors, inter-rater agreement, or whether idea novelty was checked against the full prior literature. If the reviewers have a conflict of interest, the human evaluation in Table 10 is weakened.

how reviews work

0 comments
Cite this review

Pith. "Pith review of InternAgent: When Agent Becomes the Scientist -- Building Closed-Loop System from Hypothesis to Verification." pith.science (2026). https://pith.science/paper/24TDR375

@misc{pith2026250516938,
  author       = {Pith},
  title        = {Pith review of: InternAgent: When Agent Becomes the Scientist -- Building Closed-Loop System from Hypothesis to Verification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/24TDR375}},
  note         = {Machine review of arXiv:2505.16938}
}
read the original abstract

Artificial Intelligence (AI) is accelerating the transformation of scientific research paradigms, not only enhancing research efficiency but also driving innovation. We introduce InternAgent, a unified closed-loop multi-agent framework to conduct Autonomous Scientific Research (ASR) across various scientific research fields, enabling researchers to tackle complicated problems in these fields with unprecedented speed and precision. InternAgent highlights three key advantages: 1) Scalability: InternAgent has demonstrated its versatility across 12 scientific research tasks, capable of generating innovative ideas to enhance the performance of baseline code. 2) Interactivity: InternAgent provides an interface for human expert feedback and multi-agent interaction in automated end-to-end processes, allowing for the seamless integration of domain expert knowledge. 3) Efficiency: InternAgent has achieved promising performance gains in several scientific fields with significantly less time cost compared to human efforts. For instance, in reaction yield prediction, it increased from 27.6% to 35.4% in just 12 hours; in enhancer activity prediction, accuracy rose from 0.65 to 0.79 with only 4 hours of processing; and in 2D semantic segmentation, precision advanced from 78.8% to 81.0% in a mere 30 hours.

Figures

Figures reproduced from arXiv: 2505.16938 by the authors.

Figure 1
Figure 1. InternAgent can support 12 types of scientific research tasks ranging from the AI field to the science field, including reaction yield prediction, molecular dynamics, power flow estimation, time series forecasting, transcription prediction, enhancer activity predic￾tion, sentiment classification, 2D image classification, 3D point classification, 2D semantic segmentation, 3D autonomous driving, large vision-language … view at source ↗
Figure 2
Figure 2. InternAgent covers three main capabilities: 1) Self-evolving Idea Generation with Human-interactive Feedback, 2) Idea-to-Methodology Construction, and 3) Evolutionary Experimental Planning and Execution. 2.1 Self-Evolving Idea Generation with Human-interactive Feedback The self-evolving idea generation capability is central to InternAgent, enabling the frame￾work to autonomously generate and refine innovative resear… view at source ↗
Figure 3
Figure 3. InternAgent Self-evolutionary path of ideas for reaction yield prediction task. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Analysis of two modes on survey agent. (i.e., Init Idea 0) denotes an initially generated idea and the child nodes are evolved from the parent node. As ideas continue to evolve, more external knowledge sourced from the survey agent is incorporated into ideas, which enr…
Figure 5
Figure 5. Figure 5: Visual Examples of AutoRYP Task. 4 Case Studies 4.1 Qualitative Results In this section, we present the results from various tasks implemented using InternAgent. 4.1.1 Visual Examples of Various Tasks We present showcases for three distinct tasks: AutoRYP, AutoMD, and …
Figure 6
Figure 6. Figure 6: Visual Examples of AutoMD Task. attention modules, this system not only combines 1D and 2D molecular representations effectively but also improves prediction accuracy and model adaptability. Consequently, this showcase underscores the method’s potential for advancing r…
Figure 7
Figure 7. Figure 7: Visual Examples of Experimental Planning and Adaptive Evolution on Auto3DCls [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Visual Examples of Experimental Planning and Adaptive Evolution on AutoTPPR [PITH_FULL_IMAGE:figures/full_fig_p030_8.png]
Figure 9
Figure 9. Figure 9: Result Analysis (Generated Idea, Method, and Code) of AutoPower Task [PITH_FULL_IMAGE:figures/full_fig_p031_9.png]
Figure 10
Figure 10. Figure 10: Result Analysis (Generated Idea, Method, and Code) of AutoSenCls Task [PITH_FULL_IMAGE:figures/full_fig_p032_10.png]
Figure 11
Figure 11. Figure 11: Result Analysis (Generated Idea, Method, and Code) of Auto3DCls Task [PITH_FULL_IMAGE:figures/full_fig_p033_11.png]
Figure 12
Figure 12. Figure 12: Result Analysis (Generated Idea, Method, and Code) of Auto2DCls Task [PITH_FULL_IMAGE:figures/full_fig_p034_12.png]
Figure 13
Figure 13. Figure 13: InternAgent software platform includes features such as the user entry interface, [PITH_FULL_IMAGE:figures/full_fig_p034_13.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SeekBrain: An Autonomous Multi-Agent System for Accelerating Neuroscience Discovery

    cs.MA 2026-07 conditional novelty 6.0 of 10

    SeekBrain, a multi-agent system with a neuroscience analysis recipe library, outperforms Claude Code and Codex on 32 expert-scored neuroscience analysis tasks and carries out two published-data analyses.

  2. Structural Enforcement of Statistical Rigor in AI-Driven Discovery: A Functional Architecture

    cs.SE 2025-11 conditional novelty 6.0 of 10

    An AI-Scientist guard architecture combining a Haskell monad for online FDR accounting with declarative scaffolding against data leakage; simulation supports it, but the advertised Lean/SPARK verification is absent fr...

  3. SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning

    cs.CV 2025-08 conditional novelty 6.0 of 10

    An RL framework that trains vision-language models to self-correct captions via a scene-graph-based reward outperforms SFT and DPO on caption quality.

  4. Rethinking Scientific Discovery in the Agentic Era

    cs.CL 2026-07 conditional novelty 5.5 of 10

    SCION claims an agentic OS with Research Execution Plans and layered memory that beats autonomous research-agent baselines on reading, ideation, molecule design, and antibody screening.

  5. SciDER: Scientific Data-centric End-to-end Researcher

    cs.AI 2026-03 unverdicted novelty 5.0 of 10

    SciDER is a data-centric multi-agent system that automates ideation, raw-data analysis, experiment coding, and critique, with reported leading results on six scientific-agent benchmarks.

  6. AI for Auto-Research: Roadmap & User Guide

    cs.AI 2026-05 unverdicted novelty 4.0 of 10

    The paper delivers a stage-by-stage roadmap for AI in research, showing reliable assistance in retrieval and tool tasks but fragility in novelty and judgment, advocating human-governed collaboration.

  7. A Vision for Geo-Temporal Deep Research Systems: Towards Comprehensive, Transparent, and Reproducible Geo-Temporal Information Synthesis

    cs.CL 2025-06 conditional novelty 4.0 of 10

    A research agenda calling for geo-temporal reasoning in deep research systems, with no experiments or system implementation.

Reference graph

Works this paper leans on

55 extracted references · 20 canonical work pages · cited by 7 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  3. [3]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  4. [4]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    work page

  5. [5]

    Genome-wide quantitative enhancer activity maps identified by starr-seq

    Cosmas D Arnold, Daniel Gerlach, Christoph Stelzer, ukasz M Bory \'n , Martina Rath, and Alexander Stark. Genome-wide quantitative enhancer activity maps identified by starr-seq. Science, 339 0 (6123): 0 1074--1077, 2013

    work page 2013

  6. [6]

    Encoder-decoder with atrous separable convolution for semantic image segmentation

    Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pp.\ 801--818, 2018

    work page 2018

  7. [7]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. Science China Information Sciences, 67 0 (12): 0 220101, 2024

    work page 2024

  8. [8]

    u tt, and Klaus-Robert M \

    Stefan Chmiela, Alexandre Tkatchenko, Huziel E Sauceda, Igor Poltavsky, Kristof T Sch \"u tt, and Klaus-Robert M \"u ller. Machine learning of accurate energy-conserving molecular force fields. Science advances, 3 0 (5): 0 e1603015, 2017

    work page 2017

Show all 55 references
  1. [9]

    Deepstarr predicts enhancer activity from dna sequence and enables the de novo design of synthetic enhancers

    Bernardo P de Almeida, Franziska Reiter, Michaela Pagani, and Alexander Stark. Deepstarr predicts enhancer activity from dna sequence and enables the de novo design of synthetic enhancers. Nature genetics, 54 0 (5): 0 613--624, 2022

    2022

  2. [10]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics , pp.\ 4171...

    2019

  3. [11]

    Everingham, L

    M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL V isual O bject C lasses C hallenge 2012 (VOC2012) R esults. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html, 2012

    2012

  4. [12]

    Aider: Ai pair programming in your terminal

    Paul Gauthier and Aider-AI Contributors. Aider: Ai pair programming in your terminal. https://github.com/Aider-AI/aider, 2023. URL https://github.com/Aider-AI/aider. Accessed: 2025-05-07

    2023

  5. [13]

    Towards an ai co-scientist

    Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Anil Palepu, Petar Sirkovic, Artiom Myaskovsky, Felix Weissenberger, Keran Rong, Ryutaro Tanno, et al. Towards an ai co-scientist. arXiv preprint arXiv:2502.18864, 2025

    2025 arXiv

  6. [14]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  7. [15]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  8. [16]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  9. [17]

    Ai-researcher: Fully-automated scientific discovery with llm agents

    HKU Data Intelligence Lab. Ai-researcher: Fully-automated scientific discovery with llm agents. https://github.com/HKUDS/AI-Researcher, 2025. URL https://github.com/HKUDS/AI-Researcher

    2025

  10. [18]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024 a

    2024 arXiv

  11. [19]

    Chain of ideas: Revolutionizing research in novel idea development with llm agents

    Long Li, Weiwen Xu, Jiayan Guo, Ruochen Zhao, Xinxuan Li, Yuqian Yuan, Boqiang Zhang, Yuming Jiang, Yifei Xin, Ronghao Dang, et al. Chain of ideas: Revolutionizing research in novel idea development with llm agents. arXiv preprint arXiv:2410.13185, 2024 b

    arXiv

  12. [20]

    The ai scientist: Towards fully automated open-ended scientific discovery

    Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292, 2024

    2024 arXiv

  13. [21]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255, 2023

    arXiv

  14. [22]

    Ursa: Understanding and verifying chain-of-thought reasoning in multimodal mathematics

    Ruilin Luo, Zhuofan Zheng, Yifan Wang, Yiyao Yu, Xinzhe Ni, Zicheng Lin, Jin Zeng, and Yujiu Yang. Ursa: Understanding and verifying chain-of-thought reasoning in multimodal mathematics. arXiv preprint arXiv:2501.04686, 2025

    2025

  15. [23]

    One million scenes for autonomous driving: Once dataset

    Jiageng Mao, Minzhe Niu, Chenhan Jiang, Hanxue Liang, Jingheng Chen, Xiaodan Liang, Yamin Li, Chaoqiang Ye, Wei Zhang, Zhenguo Li, et al. One million scenes for autonomous driving: Once dataset. arXiv preprint arXiv:2106.11037, 2021

    2021 arXiv

  16. [24]

    Exploring genetic interaction manifolds constructed from rich single-cell phenotypes

    Thomas M Norman, Max A Horlbeck, Joseph M Replogle, Alex Y Ge, Albert Xu, Marco Jost, Luke A Gilbert, and Jonathan S Weissman. Exploring genetic interaction manifolds constructed from rich single-cell phenotypes. Science, 365 0 (6455): 0 786--793, 2019

    2019

  17. [25]

    A platform for automated nanomole-scale reaction screening and micromole-scale synthesis in flow

    Damith Perera, Joseph W Tucker, Shalini Brahmbhatt, Christopher J Helal, Ashley Chong, William Farrell, Paul Richardson, and Neal W Sach. A platform for automated nanomole-scale reaction screening and micromole-scale synthesis in flow. Science, 359 0 (6374): 0 429--434, 2018

    2018

  18. [26]

    Large language models are zero shot hypothesis proposers

    Biqing Qi, Kaiyan Zhang, Haoxiang Li, Kai Tian, Sihang Zeng, Zhang-Ren Chen, and Bowen Zhou. Large language models are zero shot hypothesis proposers. arXiv preprint arXiv:2311.05965, 2023

    2023 arXiv

  19. [27]

    Large language models as biomedical hypothesis generators: a comprehensive evaluation

    Biqing Qi, Kaiyan Zhang, Kai Tian, Haoxiang Li, Zhang-Ren Chen, Sihang Zeng, Ermo Hua, Hu Jinfang, and Bowen Zhou. Large language models as biomedical hypothesis generators: a comprehensive evaluation. arXiv preprint arXiv:2407.08940, 2024

    2024 arXiv

  20. [28]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 652--660, 2017

    2017

  21. [29]

    Ai idea bench 2025: Ai research idea generation benchmark

    Yansheng Qiu, Haoquan Zhang, Zhaopan Xu, Ming Li, Diping Song, Zheng Wang, and Kaipeng Zhang. Ai idea bench 2025: Ai research idea generation benchmark. arXiv preprint arXiv:2504.14191, 2025

    2025 arXiv

  22. [30]

    Predicting transcriptional outcomes of novel multigene perturbations with gears

    Yusuf Roohani, Kexin Huang, and Jure Leskovec. Predicting transcriptional outcomes of novel multigene perturbations with gears. Nature Biotechnology, 42 0 (6): 0 927--935, 2024

    2024

  23. [31]

    Agentrxiv: Towards collaborative autonomous research

    Samuel Schmidgall and Michael Moor. Agentrxiv: Towards collaborative autonomous research. arXiv preprint arXiv:2503.18102, 2025

    2025 arXiv

  24. [32]

    Agent laboratory: Using llm agents as research assistants

    Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. Agent laboratory: Using llm agents as research assistants. arXiv preprint arXiv:2501.04227, 2025

    2025 arXiv

  25. [33]

    Can llms generate novel research ideas? a large-scale human study with 100+ nlp researchers

    Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can llms generate novel research ideas? a large-scale human study with 100+ nlp researchers. arXiv preprint arXiv:2409.04109, 2024

    2024 arXiv

  26. [34]

    Recursive deep models for semantic compositionality over a sentiment treebank

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language proce...

    2013

  27. [35]

    Openpcdet: An open-source toolbox for 3d object detection from point clouds

    OpenPCDet Development Team. Openpcdet: An open-source toolbox for 3d object detection from point clouds. https://github.com/open-mmlab/OpenPCDet, 2020

    2020

  28. [36]

    Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024 a

    2024 arXiv

  29. [37]

    Scimon: Scientific inspiration machines optimized for novelty

    Qingyun Wang, Doug Downey, Heng Ji, and Tom Hope. Scimon: Scientific inspiration machines optimized for novelty. arXiv preprint arXiv:2305.14259, 2023

    2023 arXiv

  30. [38]

    Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brenna...

    2024 arXiv

  31. [39]

    Openhands: An open platform for ai software developers as generalist agents

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, et al. Openhands: An open platform for ai software developers as generalist agents. In The Thirteenth International Conference on Learning Representa...

    2024

  32. [40]

    Enhancing geometric representations for molecules with equivariant vector-scalar interactive message passing

    Yusong Wang, Tong Wang, Shaoning Li, Xinheng He, Mingyu Li, Zun Wang, Nanning Zheng, Bin Shao, and Tie-Yan Liu. Enhancing geometric representations for molecules with equivariant vector-scalar interactive message passing. Nature Communications, 15 0 (1): 0 313, 2024 d

    2024

  33. [41]

    3d shapenets: A deep representation for volumetric shapes

    Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 1912--1920, 2015

    1912

  34. [42]

    The ai scientist-v2: Workshop-level automated scientific discovery via agentic tree search

    Yutaro Yamada, Robert Tjarko Lange, Cong Lu, Shengran Hu, Chris Lu, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist-v2: Workshop-level automated scientific discovery via agentic tree search. arXiv preprint arXiv:2504.08066, 2025

    2025 arXiv

  35. [43]

    Surveyforge: On the outline heuristics, memory-driven generation, and multi-dimensional evaluation for automated survey writing

    Xiangchao Yan, Shiyang Feng, Jiakang Yuan, Renqiu Xia, Bin Wang, Bo Zhang, and Lei Bai. Surveyforge: On the outline heuristics, memory-driven generation, and multi-dimensional evaluation for automated survey writing. arXiv preprint arXiv:2503.04629, 2025

    2025 arXiv

  36. [44]

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement, 2024b. URL https://arxiv. org/abs/2409.12122

    arXiv

  37. [45]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  38. [46]

    Large language models for automated open-domain scientific hypotheses discovery

    Zonglin Yang, Xinya Du, Junxian Li, Jie Zheng, Soujanya Poria, and Erik Cambria. Large language models for automated open-domain scientific hypotheses discovery. arXiv preprint arXiv:2309.02726, 2023

    2023 arXiv

  39. [47]

    Center-based 3d object detection and tracking

    Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center-based 3d object detection and tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 11784--11793, 2021

    2021

  40. [48]

    Dolphin: Closed-loop open-ended auto-research through thinking, practice, and feedback

    Jiakang Yuan, Xiangchao Yan, Botian Shi, Tao Chen, Wanli Ouyang, Bo Zhang, Lei Bai, Yu Qiao, and Bowen Zhou. Dolphin: Closed-loop open-ended auto-research through thinking, practice, and feedback. arXiv preprint arXiv:2501.03916, 2025

    2025 arXiv

  41. [49]

    Wide residual networks

    Sergey Zagoruyko. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016

    2016 arXiv

  42. [50]

    Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp.\ 11121--11128, 2023

    Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp.\ 11121--11128, 2023

    2023

  43. [51]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 11975--11986, 2023

    2023

  44. [52]

    Senseflow: A physics-informed and self-ensembling iterative framework for power flow estimation, 2024

    Zhen Zhao, Zhen Huang, Zicheng Wang, Wenqi Huang, and LEI BAI. Senseflow: A physics-informed and self-ensembling iterative framework for power flow estimation, 2024. URL https://openreview.net/forum?id=UKiCFpwcqY

    2024

  45. [53]

    Informer: Beyond efficient transformer for long sequence time-series forecasting

    Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In The Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Conference , volu...

    2021

  46. [54]

    Hypothesis generation with large language models

    Yangqiaoyu Zhou, Haokun Liu, Tejes Srivastava, Hongyuan Mei, and Chenhao Tan. Hypothesis generation with large language models. arXiv preprint arXiv:2404.04326, 2024

    2024 arXiv

  47. [55]

    Matpower: Steady-state operations, planning, and analysis tools for power systems research and education

    Ray Daniel Zimmerman, Carlos Edmundo Murillo-S \'a nchez, and Robert John Thomas. Matpower: Steady-state operations, planning, and analysis tools for power systems research and education. IEEE Transactions on power systems, 26 0 (1): 0 12--19, 2010

    2010

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.