REVIEW 12 cited by
ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large Language Models (LLMs) still struggle with natural language reasoning tasks. Motivated by the society of minds (Minsky, 1988), we propose ReConcile, a multi-model multi-agent framework designed as a round table conference among diverse LLM agents. ReConcile enhances collaborative reasoning between LLM agents via multiple rounds of discussion, learning to convince other agents to improve their answers, and employing a confidence-weighted voting mechanism that leads to a better consensus. In each round, ReConcile initiates discussion between agents via a 'discussion prompt' that consists of (a) grouped answers and explanations generated by each agent in the previous round, (b) their confidence scores, and (c) demonstrations of answer-rectifying human explanations, used for convincing other agents. Experiments on seven benchmarks demonstrate that ReConcile significantly improves LLMs' reasoning -- both individually and as a team -- surpassing prior single-agent and multi-agent baselines by up to 11.4% and even outperforming GPT-4 on three datasets. ReConcile also flexibly incorporates different combinations of agents, including API-based, open-source, and domain-specific models, leading to an 8% improvement on MATH. Finally, we analyze the individual components of ReConcile, demonstrating that the diversity originating from different models is critical to its superior performance. Code: https://github.com/dinobby/ReConcile
Forward citations
Cited by 12 Pith papers
-
Small, Free, and Effective: Orchestrating Open-Weight Small Language Models to Outperform Single LLM for Malware Analysis
An evidence-grounded hybrid of 4B and 8B open-weight models reaches 35.30% on malware-report QA, slightly beating the best ungrounded frontier model (34.77%) while trailing a frontier model given the same evidence (38.22%).
-
A global log for medical AI
MedLog defines a nine-field, syslog-style event log for clinical AI, intended to support real-world surveillance and auditing; the four-deployment validation claimed in the abstract is absent from the body.
-
Evaluating Large Language Models as Expert Annotators
Material Fingerprinting recovers the form and parameters of hyperelastic material models by nearest-neighbor matching of test data against a simulated fingerprint database: exact at zero noise, degrading under 5% noise.
-
DynamiCare: A Dynamic Multi-Agent Framework for Interactive and Open-Ended Medical Decision-Making
DynamiCare is a multi-agent LLM framework that runs multi-round diagnostic dialogues with a dynamically adjusted specialist team, evaluated on a new 500-patient benchmark built from MIMIC-III.
-
Decision Protocols in Multi-Agent Large Language Model Conversations
Consensus decision protocols beat voting/judge on knowledge QA for Llama-3 multi-agent chats, while voting and judge win on logic tasks; independent initial drafts raise accuracy and extra voting-time info barely helps.
-
ACD-CLIP: Decoupling Representation and Dynamic Fusion for Zero-Shot Anomaly Detection
ACD-CLIP improves zero-shot anomaly detection by co-designing a convolutional low-rank adapter with a dynamic fusion gateway that modulates text prompts from visual context.
-
How to Train a Leader: Hierarchical Reasoning in Multi-Agent LLMs
A leader LLM trained with a GRPO variant that conditions on frozen agent responses improves both collaborative and zero-shot accuracy on BBH, MATH, and MMLU.
-
Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferences
LLM agents can run a simulated decision conference, and a dedicated agreement-detection agent helps the debate cover topics that match a real expert workshop.
-
CortexDebate: Debating Sparsely and Equally for Multi-Agent Debate
CortexDebate prunes the multi-agent debate graph every round using a McKinsey-style trust score per directed link, reporting accuracy gains over full-debate baselines on eight datasets with shorter per-agent contexts.
-
An Adversary-Resistant Multi-Agent LLM System via Credibility Scoring
A credibility-scoring framework for multi-agent LLM systems, learning agent trustworthiness on the fly and weighting outputs accordingly, improves accuracy under adversarial conditions in some benchmarks.
-
LightRouter: Towards Efficient LLM Collaboration with Minimal Overhead
LightRouter uses short preview outputs to filter a pool of LLMs down to two, then aggregates their full responses, beating ensemble baselines and matching costlier models.
-
SIGMA: Search-Augmented On-Demand Knowledge Integration for Agentic Mathematical Reasoning
SIGMA uses four specialized retrieval agents plus a moderator to improve math reasoning, reporting up to 7.4 absolute MATH500 points over Search-o1 at 1.5B scale.
Discussion (0). Sign in to comment.