REVIEW 14 cited by
MARG: Multi-Agent Review Generation for Scientific Papers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We study the ability of LLMs to generate feedback for scientific papers and develop MARG, a feedback generation approach using multiple LLM instances that engage in internal discussion. By distributing paper text across agents, MARG can consume the full text of papers beyond the input length limitations of the base LLM, and by specializing agents and incorporating sub-tasks tailored to different comment types (experiments, clarity, impact) it improves the helpfulness and specificity of feedback. In a user study, baseline methods using GPT-4 were rated as producing generic or very generic comments more than half the time, and only 1.7 comments per paper were rated as good overall in the best baseline. Our system substantially improves the ability of GPT-4 to generate specific and helpful feedback, reducing the rate of generic comments from 60% to 29% and generating 3.7 good comments per paper (a 2.2x improvement).
Forward citations
Cited by 14 Pith papers
-
Does Multi-Agent Debate Improve AI Feedback on Research Papers?
Authors of economics meta-analyses found a single-pass AI report more useful than two multi-agent debate tools that cost up to thirty times more to run.
-
AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Research
A benchmark of 1,500 expert-annotated ablation study designs from 807 NLP papers shows frontier LLMs underperform human experts and that LLM-as-a-judge evaluations correlate weakly with human judgments.
-
AutoSupervision: Closing the Feedback Loop in Scientific Workflows with Grounded Revision Verification
On a new 8,790-instance benchmark built from Nature Communications review records, LLMs characterize reviewer concerns well (GPT-5.5: 0.754) but verify evidence-backed revision resolution poorly (best 0.501).
-
Agentic Re-Casting using Agentic Re-Simulations
An agentic AI system with a physicist in the loop re-casts an ATLAS ttZ measurement into a global top-quark SMEFT fit and recovers injected coloron Wilson coefficients in a repeatable benchmark.
-
Do Methods Support the Claims? Intra-Paper Verification for Peer Review
Introduces intra-paper claim verification, a GPT-4o pipeline that extracts novelty claims from a paper's introduction and checks whether the methods section substantiates them, validated against human reviewer concern...
-
RubricReviewer: From Direct Critique to Objective and Comprehensive Rubric-Driven Peer Review
A rubric-first LLM pipeline that splits peer review into rubric generation, rubric-conditioned review writing, and final scoring outperforms existing AI reviewers on alignment with human judgments in a 200-paper test.
-
AI Can Learn Scientific Taste
Reinforcement learning on citation-preference pairs teaches a model to predict which papers will be cited more and to propose ideas that LLM judges rate as likely to be cited more—but "taste" here means citation impact.
-
MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detection
MIND uses unlabeled similar memes, bidirectional AI insight derivation, and multi-agent debate to improve zero-shot harmful meme detection on HarM, FHM, and MAMI.
-
Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers
LIMITGEN evaluates LLMs on identifying paper limitations and shows strong models still miss roughly half of obvious flaws, with retrieval augmentation giving modest but consistent gains.
-
Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Models
A new benchmark (TruthHypo) and a knowledge-grounded hallucination detector (KnowHD) show that grounding scores can partially select truthful LLM-generated biomedical hypotheses, but the result is at risk from knowled...
-
AI for Auto-Research: Roadmap & User Guide
The paper delivers a stage-by-stage roadmap for AI in research, showing reliable assistance in retrieval and tool tasks but fragility in novelty and judgment, advocating human-governed collaboration.
-
How Far Are AI Scientists from Changing the World?
This survey proposes a four-level capability framework for AI Scientist systems and, using an AI reviewer, finds that current systems produce papers rated well below normal scientific standards.
-
AI4Research: A Survey of Artificial Intelligence for Scientific Research
A survey that organizes AI-for-research work into five tasks, comprehension, survey, discovery, writing, and peer review, and compiles associated tools and benchmarks.
-
OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Models
The paper advocates protecting and leveraging OpenReview's peer review corpus as a community asset for LLM-based review assistance, benchmarks, and alignment.
Discussion (0). Sign in to comment.