Pith. sign in

REVIEW 3 major objections 4 minor 20 references

IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Treating research ideation as a quality-diversity search, with lineage archives and targeted repair, yields four times more pursuable, mutually distinct ideas than the best baseline.

desk verdict Genuinely new QD-for-ideation formulation with a solid engineering story, but the abstract's headline ratios overstate the results; the qualitative direction holds. read the letter →

arxiv 2607.22375 v1 pith:I4ORJ62L submitted 2026-07-24 cs.AI

Varun Gumma , Navonil Majumder , Soumitra Sinhahajari , Soujanya Poria This is my paper · ORCID
classification cs.AI
keywords Quality-DiversitysearchresearchideagenerationLLMagentsscientificdiscoveryrepairandrefinementlineagearchivesyieldmetricnovelty-diversitytradeoff
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LLM-based research ideation has been treated as an isolated optimization problem for either quality or diversity, which produces near-duplicate ideas or trivial proposals. This paper argues that ideation should be a Quality-Diversity (QD) search: the goal is a set of ideas that are each non-obvious, sound, and clear, while remaining mutually distinct. The authors build IDEAgent, a multi-agent framework that evolves ideas through lineages, keeps compact summaries of active, historical, and rejected ideas, and repairs or refines drafts that miss thresholds. They also introduce Yield, a joint metric that counts the largest set of mutually diverse ideas passing quality thresholds. Across 32 computer-science topics, IDEAgent achieves 3.89x the Yield of the best baseline at the strictest gate and produces at least one qualifying idea on 27 of 32 topics, versus 8 for the baseline.

What carries the argument

The load-bearing machinery is the lineage-archive search loop paired with the Yield metric. Each idea is condensed into a five-field signature (problem, mechanism, value-add, assumptions, expected effect); a controller routes candidates through four cases — pure historical match, single-lineage match, below-diversity-floor, or eligible — before one repair or two refinements are allowed. The qualification gate requires non-obviousness, soundness, and clarity scores of at least 60 on a 0–100 scale, no invalid or disputed soundness judgment, and a diversity floor of 60 relative to all archives. Yield operationalizes the QD conjunction: it filters ideas by quality thresholds, then extracts the m

What would settle it

Have domain experts independently score a sample of the IDEAgent-generated ideas on the same rubrics and recompute Yield with human scores; if the human-based Yield no longer shows IDEAgent ahead of the baselines — or if repair or refinement lowers human-rated soundness — the central claim collapses. A lighter check: run the pipeline on a benchmark where idea soundness can be verified computationally (e.g., known results), and compare judge scores to ground truth.

Watch

Extended reading notes

Core claim

The central discovery is that the conjunction of quality and diversity — enforced by a lineage-archive search with targeted repair and refinement — is what converts a fixed generation budget into a dense portfolio of distinct, viable research directions. IDEAgent's controller maintains active, historical, and rejected archives, comparing each new idea's compact signature against all prior generations before deciding whether to accept, repair, refine, or reject it. The paper's quantitative claim is that this pipeline, evaluated with the new Yield metric, achieves a mean Yield of 1.094 at the gate NB≥7, S≥7, C≥6, D≥7, compared with 0.281 for the NOVA-inspired baseline — a 3.89x improvement — a

Load-bearing premise

The evaluation rests entirely on LLM judges' scores for non-obviousness, soundness, clarity, and pairwise diversity — both the internal gating that decides repair, refinement, and rejection, and the external Yield that reports success; the paper's own inter-judge agreement on soundness is only κ=0.268, and no human validation is provided.

Editorial extensions

If this is right

  • A fixed ideation budget — ten fresh seeds plus at most two auxiliary drafts — can be converted into roughly four times as many distinct, high-quality ideas as independent generation, and the gain grows at stricter non-obviousness thresholds.
  • Removing the repair/refinement loop (the Sequential-Memory baseline) drops Yield at the strictest gate from 1.094 to 0.281, so the quality-improvement subroutine is the main driver of the advantage.
  • Yield is model-agnostic and can be recomputed with any judge, including human evaluators, so it offers a game-resistant way to measure the quality-diversity tradeoff in any open-ended generation system.
  • Because the framework relies on compact signatures rather than full texts, it scales sub-linearly in memory and cost, and the authors argue the same principles should transfer to future models.
  • Cost per successful diverse portfolio is lower for IDEAgent than for the strongest baseline when at least two diverse qualifying ideas are required, and the gap widens with three.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: Because both internal routing and external Yield depend on LLM judges, the 3.89x advantage could shrink or vanish under human expert review if the judges reward stylistic polish rather than genuine logical rigor. A human-scored replication on a sample of the 320 ideas would settle this.
  • Editorial: The lineage-archive mechanism — logging active, historical, and rejected ideas as compact signatures — mirrors how human researchers record dead ends and superseded hypotheses; the same pattern could generalize to other open-ended search loops such as experiment design or automated molecule discovery.
  • Editorial: The maximum-clique Yield metric could be adopted as a general-purpose evaluation for generation tasks that value both quality and coverage, potentially replacing averaged novelty scores that near-duplicate outputs can game.
  • Editorial: The finding that one-shot generation achieved zero Yield suggests that fully shared context during generation contaminates ideas; future systems may need a controlled information flow between generations, a direction the paper leaves implicit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes IDEAgent, an LLM-based multi-agent system that frames research ideation as a Quality-Diversity (QD) search. IDEAgent generates seed ideas, evaluates them along non-obviousness, soundness, clarity, and diversity using LLM judges, and conditionally applies repair/refinement while maintaining lineage archives and compact memories. The authors also introduce Yield, a set-level metric that counts the largest subset of ideas passing quality thresholds and pairwise diversity constraints. They evaluate on 32 CS topics against four baselines (Stateless, One-Shot, Sequential-Memory, NOVA), reporting that IDEAgent outperforms the best baseline by 3.89x on Yield and achieves non-zero Yield on 8x more topics, with quality improvements attributed to repair/refinement. The code, rubrics, and per-judge tables are included in the paper and appendix.

Significance. If validated, the paper makes a useful contribution: formulating ideation as QD search with lineage-based archives, a joint Yield metric, and an open-source implementation are all practical assets. The appendix's per-judge results, cost analysis, and scaling experiments are welcome. However, the evaluation is entirely LLM-judged and the headline quantitative claims overstate what the paper's own tables show. The qualitative direction (IDEAgent > baselines) survives per-judge inspection, but the reported magnitudes are artifacts of a particular pooling choice and are not robust across judges. With corrected reporting and at least minimal human calibration, the contribution would be publishable; in its current form the evidence does not support the abstract's precise claims.

major comments (3)
  1. [Abstract, §7, Tables 1, 12, 13] The claimed 3.89x and 8x improvements are not supported by the per-judge tables. At the headline gate (D≥7, S≥7, C≥6, NB≥7), Table 1 gives Yield 1.094 vs 0.281, but this is a 'jury' result obtained by averaging the two judges' per-idea scores before thresholding. Tables 12 and 13 show per-judge ratios of 3.531/1.562 = 2.26x (Claude Opus) and 1.219/0.500 = 2.44x (Claude Sonnet). Similarly, '8x more topics' is not the non-zero-Yield ratio: Table 1 at Yield≥1 shows 27/32 vs 8/32 = 3.4x; 8x appears only at Yield≥2 (8/32 vs 1/32), and even there per-judge ratios are 1.8x and 5x. The abstract and §7 should report per-judge ranges and unambiguously define the jury averaging procedure.
  2. [§5, §9 (Limitations), Table 2] The central evaluation is entirely LLM-judged. Both the internal qualification/repair decisions and the external Yield metric rely on model scores, with no human expert ratings. Soundness has linear-weighted κ=0.268 and Spearman ρ=0.409 between the two external judges; hard Yield thresholds can amplify judge disagreement into large differences. The jury Yield is much lower than either judge's individual Yield (e.g., NOVA: 0.281 vs 1.562 and 0.500), showing the method's advantage is partly a product of averaging near-threshold scores. I request either (a) a human-evaluated subset of 40-60 ideas with expert scores and Yield recomputed on those scores, or (b) an explicit re-scoping of all conclusions to 'according to the chosen LLM judges' with disagreement-aware intervals. The paper is transparent about this limitation, but it remains load-bearing for the central quantitative claim.
  3. [§7, Fig. 3b] The claim that repair/refinement are 'crucial' is based on small external-judge deltas. For example, repair raises external soundness from 6.39 to 6.88 on a 0-9 scale using only 30 lineages, and refinement raises Yield by +0.78 to +0.88. These differences are of the same order as the inter-judge soundness disagreement (κ=0.268), and no confidence intervals or effect sizes are reported. The internal scores in Fig. 3a are on a 0-100 scale and cannot be compared directly. The paper should report uncertainty (e.g., bootstrap CIs across topics) and paired tests for the external repair/refinement deltas, or temper the claim.
minor comments (4)
  1. [Title/header] The header spells the system as 'IDEAAgent' while the abstract and body use 'IDEAgent'; please standardize.
  2. [Table 3] 'Scaling LLM Reasoning via Reinforcement Learning' is listed twice under cs.AI. If these are two distinct topic instances they should be distinguished; otherwise the dataset contains fewer than 32 unique topics.
  3. [Tables 1, 12, 13] The relationship between Table 1's jury Yield and the per-judge Yields should be defined explicitly (per-idea score averaging before thresholding) so readers are not misled by the seemingly inconsistent means across tables.
  4. [Figures 2 and 4] The color scale and axis labels are small; also the caption 'Claude-only two-judge Yield surface' is confusing since two Claude models are used, not a single judge. Clarify.

Circularity Check

2 steps flagged · score 5.0 of 10

Headline Yield ratios are constructed from the paper's own averaged LLM-jury scores and the paper's own Yield definition; the two headline numbers do not match the paper's per-judge tables, and the evaluation stack is self-referential (same-team rubrics, LLM judges for both gating and final metric), though not definitionally forced.

  1. fitted input called prediction [Abstract; §6.1 Agents; §7 Results and Analysis; Table 1]
    "we show that IDEAgent outperforms the best baseline by 3.89x on Yield, while achieving non-zero Yield on 8x more topics. ... we employ two external large-scale models, claude-sonnet-57 ... and claude-opus-4.77 ... for all judgments. During analysis, we average the scores from both judges and treat the result as a single jury verdict"

    The headline 3.89x is the mean-Yield ratio of averaged jury scores at one fixed gate (1.094/0.281 in Table 1), while the paper's own per-judge tables give 3.531/1.562 = 2.26x (Opus) and 1.219/0.500 = 2.44x (Sonnet). Averaging the two judges' per-idea scores before thresholding suppresses the baseline's marginal ideas more than IDEAgent's, so the advertised ratio is an artifact of the averaging protocol rather than a stable measured property. Similarly, the '8x more topics' claim is only true at Yield>=2 (8/32 vs 1/32); at the natural 'non-zero Yield' reading Yield>=1, Table 1 reports 27/32 vs 8/32, a 3.4x ratio. The metric and the judges are both defined by the authors, so the headline numbers are constructed from the paper's own evaluation stack rather than independently validated quantit

  2. other [Limitations §9.1; §2 Related Work (citation [Sinhahajari et al. 2026]); §4 Background Corpus Collection]
    "All internal and external evaluations rely primarily on LLM-based judges to assess non-obviousness, soundness, clarity, feasibility, and diversity. ... we do not evaluate real-world feasibility, exact-correctness, or practical effectiveness of the generated ideas"

    The central claim that IDEAgent produces genuinely better research ideas is evaluated only by LLM judges using rubrics authored by the same team (and the topic corpus is assembled following the authors' own prior work, Sinhahajari et al. 2026). The paper's own agreement analysis shows soundness kappa = 0.268. This is not a mathematical tautology because the external judges are different model providers than the internal gate models, but the evaluation is substantially self-referential: both the internal routing (repair/refine decisions) and the external Yield metric use LLM judgments of the same constructs, with no human ground truth.

full rationale

The paper is not circular in the sense that Yield is literally defined as the pipeline's internal gate score: the external Yield is computed from two held-out Claude judges on the final 10 ideas, while IDEAgent's internal routing uses GPT/Gemini scores during generation, so the final metric is not the same function as the training signal. The methodology section is also genuinely distinct from the baselines: repair/refinement and lineage archives are new mechanisms, and the scaling experiments (Figure 5) attempt to control for generation budget. However, the advertised headline numbers are not robust to the paper's own reported per-judge tables: 3.89x appears only after averaging the two judges, and '8x more topics' only at Yield>=2 rather than at non-zero Yield. The evaluation stack is also self-referential in a weaker sense: LLM judges with the authors' rubrics are the only arbiter of both internal gating and the final metric, and the paper's own agreement data show soundness kappa = 0.268. Correctness and robustness concerns (averaging protocol, threshold sensitivity, judge agreement) are distinct from definitional circularity, so a score of 5 is appropriate: one constructed headline statistic plus a self-referential evaluation stack, but no step that reduces by construction to its own input.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The central claim rests almost entirely on the validity of LLM-as-judge evaluations, which is an unverified domain assumption. The hand-set thresholds and budget parameters are numerous, and the topic selection procedure is idiosyncratic. There are no invented physical entities, but the 'lineage' abstraction and the Yield metric are new modeling constructs introduced by the authors and used both as optimization targets and evaluation criteria.

free parameters (7)
  • Qualification thresholds τ_N=τ_S=τ_C=60, τ_D=60, δ=20 = 60/60/60/60, δ=20
    Hand-set in Section 3.3; define the gate for acceptance, repair, and diversity rejection. The evaluation headlines depend on these choices.
  • Q_b weights (0.7 NB, 0.2 S, 0.1 C) = 0.7/0.2/0.1
    Equation (5), chosen by hand; determines replacement and refinement decisions in the active archive.
  • Soft refinement targets (NB<90, S<80, C<80) = 90/80/80
    Section 3.3 Stage III; conditional trigger for refinement rather than an acceptance gate.
  • Budget B=10, active capacity C_A=10, K_aux=2 = B=10, C_A=10, K_aux=2
    Section 3.3; defines the discovery budget and number of auxiliary drafts. Affects the number of candidate ideas and the diversity of the final set.
  • Soundness panel size M=5 = 5
    Section 3.2; chosen by 'diminishing returns' rather than a principled optimization.
  • Evaluation gates (NB≥6/7, S≥7, C≥6, D≥7) = varies by table row
    Table 1 and Figures 2/4; these thresholds define Yield and the 'successful topic' counts. Headline improvements change substantially across gates.
  • Topic collection thresholds (IoU>0.5, ≥3 papers, 5–8 background papers, Γ=6) = 0.5 / 3 / 5–8 / 6
    Section 4; hand-set criteria that determine which 32 topics enter the benchmark. The topic selection could influence the relative performance of methods.
assumptions (6)
  • domain assumption LLM judges' scores are valid proxies for non-obviousness, soundness, clarity, and pairwise diversity of research ideas.
    Used throughout Sections 5–7 and acknowledged in Limitations 1. No human expert evaluation is provided; the paper's own soundness κ is only 0.268.
  • domain assumption The two external LLM judges (Claude Opus 4.7 and Claude Sonnet 5) provide a reliable 'jury verdict' when averaged.
    Section 6.2; agreement is moderate at best, and averaging two correlated model judges does not eliminate systematic bias.
  • domain assumption The background corpus construction via influential-citation IoU clustering produces meaningful research topics.
    Section 4; topics are defined by citation-overlap clusters, but there is no evidence that these clusters correspond to coherent research areas that experts would recognize.
  • domain assumption Proprietary LLMs (GPT-5.6, Gemini-3.1, etc.) have sufficient knowledge and reasoning to generate and evaluate research ideas; open-source models do not.
    Section 6.1; the paper relies on this to justify its API-based experimental setup, but does not provide independent evidence of generation or evaluation capability.
  • standard math A maximum-cardinality clique on the eligibility graph is the correct formalization of 'largest set of mutually diverse high-quality ideas'.
    Algorithm 2; this is a standard combinatorial optimization. It is tractable here because N=10, but the metric definition itself is a modeling choice.
  • domain assumption Non-obviousness can be judged without retrieval against the existing literature.
    Limitations 2 states the paper does not verify absence from existing literature, yet non-obviousness is a central quality axis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation." pith.science (2026). https://pith.science/paper/I4ORJ62L

@misc{pith2026260722375,
  author       = {Pith},
  title        = {Pith review of: IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I4ORJ62L}},
  note         = {Machine review of arXiv:2607.22375}
}
read the original abstract

Large Language Models (LLMs) have significantly automated the process of scientific discovery over the past few years. However, existing systems share one core limitation: they generate and optimize ideas independently for either Quality or Diversity. This often leads to the generation of ideas in close proximity to one another or to a large set of trivial, unsound, or unclear concepts. In this work, we instead argue that research ideation should be treated as a conjunction of both objectives and framed as a Quality-Diversity (QD) search. In line with this perspective, we introduce IDEAgent, a multi-agent framework that manages the evolution of ideas through lineages. We jointly drive Quality using multi-objective feedback for dedicated repair and refinement, while Diversity is achieved through lightweight sequential memory and explicit comparison against completed ideas, their historical ancestors, and rejected proposals. To systematically evaluate this QD conjunction, we develop Yield, a joint metric that computes the largest set of mutually diverse ideas that satisfy a predetermined quality threshold. Finally, through evaluations across 32 topics spanning 8 domains of Computer Science, we show that IDEAgent outperforms the best baseline by 3.89x on Yield, while achieving non-zero Yield on 8x more topics. We further corroborate these findings through an analysis of quality improvements, showing that repair and refinement are crucial for building logical rigor and clarity while preserving non-obviousness. To encourage future research on QD-search-based ideation, we open-source IDEAgent at https://github.com/declare-lab/IDEAgent.

Figures

Figures reproduced from arXiv: 2607.22375 by the authors.

Figure 1
Figure 1. Left: An abstract overview of IDEAgent highlighting the crucial components and flow. Right: An abstract overview of the evolution of an idea and a lineage, where a raw idea might either be directly rejected, repaired and refined, just refined, or directly accepted as per the assessment by the evaluators. Note that a refinement/repair maintains the lineage of the idea (ℓ𝐼). We purposefully omit the exact conditionals… view at source ↗
Figure 2
Figure 2. Two Judges’ average Yield for IDEAgent, Sequential-Memory baseline, and their difference. For the proportion of successful topics, we see in [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Quality improvements due to repair and refinement techniques Inter Annotator Agreement To establish the reliability of our evaluation framework, we analyze the inter-annotator agreement between the two judges. To compute the agreement per rubric as shown in [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Full Yield surfaces for IDEAgent, Sequential-Memory baseline, and their difference [PITH_FULL_IMAGE:figures/full_fig_p020_4.png]
Figure 5
Figure 5. Figure 5: Scaling fresh generation does not reproduce IDEAgent’s quality–diversity performance [PITH_FULL_IMAGE:figures/full_fig_p021_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 6 linked inside Pith

  1. [1]

    Introducing Claude Opus 4.7.https://www.anthropic.com/news/claude-opus-4-7 , April 2026a

    Anthropic. Introducing Claude Opus 4.7.https://www.anthropic.com/news/claude-opus-4-7 , April 2026a. Accessed 2026-07-16. Anthropic. Introducing Claude Sonnet 5.https://www.anthropic.com/news/claude-sonnet-5 , June 2026b. Accessed 2026-07-16. Alexis Audran-Reiss, Jordi Armengol-Estapé, Karen Hambardzumyan, Amar Budhiraja, Martin Josifoski, Edan Toledo, Ri...

    2026

  2. [3]

    ISBN 979-8-89176-189-6

    Association for Computational Linguistics. ISBN 979-8-89176-189-6. . URLhttps://aclanthology.org/2025.naacl-long.342/. Declare Lab · NTU · Under review 14 Herbie Bradley, Andrew Dai, Hannah Benita Teufel, Jenny Zhang, Koen Oostermeijer, Marco Bellagente, Jeff Clune, Kenneth Stanley, Gregory Schott, and Joel Lehman. Quality-diversity through AI feedback. I...

    2025

  3. [4]

    Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-Baptiste Mouret

    URL https://arxiv.org/abs/2507.06261. Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-Baptiste Mouret. Robots that can adapt like animals.Nature, 521(7553):503–507, May

    arXiv

  4. [8]

    XiangHu,HongyuFu,JingeWang,YifengWang,ZhikunLi,RenjunXu,YuLu,YaochuJin,LiliPan,andZhenzhong Lan

    URLhttps://arxiv.org/abs/2605.30329. XiangHu,HongyuFu,JingeWang,YifengWang,ZhikunLi,RenjunXu,YuLu,YaochuJin,LiliPan,andZhenzhong Lan. NOVA: An iterative planning framework for enhancing scientific innovation with large language models. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors,Findings of the AssociationforCom...

    arXiv 2025

  5. [11]

    Jean-Baptiste Mouret and Jeff Clune

    URLhttps://arxiv.org/abs/2408.06292. Jean-Baptiste Mouret and Jeff Clune. Illuminating search spaces by mapping elites,

    arXiv

  6. [12]

    URLhttps://arxiv.org/ abs/1504.04909. OpenAI. GPT-5.6: Frontier intelligence that scales with your ambition.https://openai.com/index/gpt-5-6/, July

    arXiv

  7. [13]

    Declare Lab · NTU · Under review 15 JustinK.Pugh,LisaB.Soros,andKennethO.Stanley

    Accessed 2026-07-16. Declare Lab · NTU · Under review 15 JustinK.Pugh,LisaB.Soros,andKennethO.Stanley. Qualitydiversity: Anewfrontierforevolutionarycomputation. Frontiers in Robotics and AI, Volume 3 - 2016,

    2026

  8. [15]

    ISBN 979-8-89176-335-7

    Association for Computational Linguistics. ISBN 979-8-89176-335-7. . URLhttps://aclanthology.org/2025.findings-emnlp.320/. Patrick Schober, Christa Boer, and Lothar A. Schwarte. Correlation coefficients: Appropriate use and interpretation. Anesthesia & Analgesia, 126(5):1763–1768,

    2025

Show all 20 references
  1. [16]

    Charles Spearman

    URLhttps://arxiv.org/abs/2606.12071. Charles Spearman. The proof and measurement of association between two things.The American Journal of Psychology, 15(1):72–101,

    arXiv

  2. [17]

    ISBN 979-8-89176-251-0

    Association for Computational Linguistics. ISBN 979-8-89176-251-0. . URLhttps://aclanthology.org/2025.acl-long. 1368/. Yixuan Tang and Yi Yang. Ai research agents narrow scientific exploration,

    2025

  3. [18]

    URLhttps://aclanthology.org/2025.sigdial-1.26/

    Association for Computational Linguistics. URLhttps://aclanthology.org/2025.sigdial-1.26/. Yutaro Yamada, Robert Tjarko Lange, Cong Lu, Shengran Hu, Chris Lu, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist-v2: Workshop-level automated scientific discovery via agent...

    2025

  4. [19]

    Keyu Zhao, Weiquan Lin, Qirui Zheng, Fengli Xu, and Yong Li

    URL https://arxiv.org/abs/2504.08066. Keyu Zhao, Weiquan Lin, Qirui Zheng, Fengli Xu, and Yong Li. Deep ideation: Designing llm agents to generate novel research ideas on scientific concept network,

    arXiv

  5. [20]

    URLhttps://arxiv.org/abs/2511.02238. A List of Domains & Topics B Evaluation: Rubrics and Algorithms C Additional Results D Cost and Futher Scaling Declare Lab · NTU · Under review 16 Table 3:List of domains and topics included in our dataset. Domain # BKDG Topic Name cs.AI 8 ...

  6. [2011]

    ISBN 9781450305570

    Association for Computing Machinery. ISBN 9781450305570. . URLhttps://doi.org/10.1145/2001576.2001606. Joel Lehman, Jonathan Gordon, Shawn Jain, Kamal Ndousse, Cathy Yeh, and Kenneth O. Stanley.Evolution Through Large Models, pages 331–366. Springer Nature Singapore, Singapore,

  7. [2015]

    URLhttps://doi.org/10.1038/nature14422

    ISSN 1476-4687. . URLhttps://doi.org/10.1038/nature14422. MatthewFontaineandStefanosNikolaidis. Differentiablequalitydiversity. InM.Ranzato,A.Beygelzimer,Y.Dauphin, P.S. Liang, and J. Wortman Vaughan, editors,Advances in Neural Information Processing Systems, volume 34, pages ...

    doi

  8. [2016]

    URLhttps://www.frontiersin.org/ journals/robotics-and-ai/articles/10.3389/frobt.2016.00040

    ISSN 2296-9144. . URLhttps://www.frontiersin.org/ journals/robotics-and-ai/articles/10.3389/frobt.2016.00040. MarissaRadensky, SimraShahid, RaymondFok, PaoSiangliulue, TomHope, andDanielS.Weld. Scideator: Human- llm compound system for scientific ideation through facet recombi...

    2016

  9. [2021]

    Google DeepMind

    URLhttps://proceedings.neurips.cc/paper_files/pap er/2021/file/532923f11ac97d3e7cb0130315b067dc-Paper.pdf. Google DeepMind. Gemini 3.1 Pro - model card.https://deepmind.google/models/model-cards/gemini-3-1-p ro/, February 2026a. Accessed 2026-07-16. Google DeepMind. Gemini 3.5...

    2021

  10. [2024]

    URL https://doi.org/10.1007/978-981-99-3814-8_11

    ISBN 978-981-99-3814-8. . URL https://doi.org/10.1007/978-981-99-3814-8_11. Long Li, Weiwen Xu, Jiayan Guo, Ruochen Zhao, Xingxuan Li, Yuqian Yuan, Boqiang Zhang, Yuming Jiang, Yifei Xin, Ronghao Dang, Yu Rong, Deli Zhao, Tian Feng, and Lidong Bing. Chain of ideas: Revolutioni...

    2025 doi

  11. [2025]

    Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang

    URLhttps://arxiv.org/abs/2511.15593. Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. ResearchAgent: Iterative research idea generation over scientific literature with large language models. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors,Proceedings o...

    2025

  12. [2026]

    URLhttps://doi.org/10.1038/s41586-026-10644-y

    ISSN 1476-4687. . URLhttps://doi.org/10.1038/s41586-026-10644-y. Sy-Tuyen Ho, Minghui Liu, Huy Nghiem, and Furong Huang. Soundnessbench: Can your ai scientist really tell good research ideas from bad ones?,

    doi

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.