REVIEW 10 cited by
Hypothesis Generation with Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Effective generation of novel hypotheses is instrumental to scientific progress. So far, researchers have been the main powerhouse behind hypothesis generation by painstaking data analysis and thinking (also known as the Eureka moment). In this paper, we examine the potential of large language models (LLMs) to generate hypotheses. We focus on hypothesis generation based on data (i.e., labeled examples). To enable LLMs to handle arbitrarily long contexts, we generate initial hypotheses from a small number of examples and then update them iteratively to improve the quality of hypotheses. Inspired by multi-armed bandits, we design a reward function to inform the exploitation-exploration tradeoff in the update process. Our algorithm is able to generate hypotheses that enable much better predictive performance than few-shot prompting in classification tasks, improving accuracy by 31.7% on a synthetic dataset and by 13.9%, 3.3% and, 24.9% on three real-world datasets. We also outperform supervised learning by 12.8% and 11.2% on two challenging real-world datasets. Furthermore, we find that the generated hypotheses not only corroborate human-verified theories but also uncover new insights for the tasks.
Forward citations
Cited by 10 Pith papers
-
Imaging-101: Benchmarking LLM Coding Agents on Scientific Computational Imaging
A 57-task, expert-verified benchmark shows frontier LLM coding agents systematically fail on physical conventions, inverse-solver choice, and end-to-end imaging pipelines.
-
AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Research
A benchmark of 1,500 expert-annotated ablation study designs from 807 NLP papers shows frontier LLMs underperform human experts and that LLM-as-a-judge evaluations correlate weakly with human judgments.
-
Formalizing Learning from Language Feedback with Provable Guarantees
Introduces a formal framework for learning from language feedback, a transfer eluder dimension complexity measure, and HELiX, a no-regret algorithm whose regret scales with this dimension.
-
HALO: Interactive Co-abductive Reasoning in Scientific Hypothesis Generation
HALO uses a three-stage co-abduction loop—clustering candidates by property improvement, distilling strategies, and synthesizing strategies—to help medicinal chemists produce more optimized and more diverse molecular ...
-
Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers
LIMITGEN evaluates LLMs on identifying paper limitations and shows strong models still miss roughly half of obvious flaws, with retrieval augmentation giving modest but consistent gains.
-
Enhancing User Engagement in Socially-Driven Dialogue through Interactive LLM Alignments
Dialogue agents aligned via DPO on preference pairs mined from simulated conversations improve engagement scores against the same simulator, with smaller and partially inconsistent human evaluation evidence.
-
Correlated Errors in Large Language Models
Large language models from different providers and architectures often make the same errors, and more accurate models are especially likely to share mistakes.
-
Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Models
A new benchmark (TruthHypo) and a knowledge-grounded hallucination detector (KnowHD) show that grounding scores can partially select truthful LLM-generated biomedical hypotheses, but the result is at risk from knowled...
-
RLIE: Rule Generation with Logistic Regression, Iterative Refinement, and Evaluation for Large Language Models
Logistic regression over LLM-generated natural-language rules outperformed letting the LLM reason with those rules on six binary text tasks.
-
InternAgent: When Agent Becomes the Scientist -- Building Closed-Loop System from Hypothesis to Verification
A closed-loop LLM-agent framework that auto-generates research ideas and code, reported to improve baseline performance on all 12 tasks it was tested on.
Discussion (0). Sign in to comment.