REVIEW 12 cited by
ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent research on Reasoning of Large Language Models (LLMs) has sought to further enhance their performance by integrating meta-thinking -- enabling models to monitor, evaluate, and control their reasoning processes for more adaptive and effective problem-solving. However, current single-agent work lacks a specialized design for acquiring meta-thinking, resulting in low efficacy. To address this challenge, we introduce Reinforced Meta-thinking Agents (ReMA), a novel framework that leverages Multi-Agent Reinforcement Learning (MARL) to elicit meta-thinking behaviors, encouraging LLMs to think about thinking. ReMA decouples the reasoning process into two hierarchical agents: a high-level meta-thinking agent responsible for generating strategic oversight and plans, and a low-level reasoning agent for detailed executions. Through iterative reinforcement learning with aligned objectives, these agents explore and learn collaboration, leading to improved generalization and robustness. Empirical results from single-turn experiments demonstrate that ReMA outperforms single-agent RL baselines on complex reasoning tasks, including competitive-level mathematical benchmarks and LLM-as-a-Judge benchmarks. Additionally, we further extend ReMA to multi-turn interaction settings, leveraging turn-level ratio and parameter sharing to improve efficiency. Comprehensive ablation studies further illustrate the evolving dynamics of each distinct agent, providing valuable insights into how the meta-thinking reasoning process enhances the reasoning capabilities of LLMs. Our code can be found in https://github.com/ziyuwan/ReMA-public
Forward citations
Cited by 12 Pith papers
-
Training Language Models to Cooperate with Inference-Time Controllers
Multi-controller post-training with turn-level GRPO over reusable reasoning modules improves LLM transfer to held-out and shifted inference-time controllers versus single-controller RL.
-
MET: Theory-Grounded and Culture-Aware Multilingual Moral Reasoning
MET-D self-distills theory-selected moral grounds into native-language reasoning, lifting macro-F1 by ~3.7–4.2 points on MCLASH and MMoralExceptQA while raising native-language chains by ~62 points.
-
Coordinating Search-Informed Reasoning and Reasoning-Guided Search in Claim Verification
A pair of reinforcement-learning-trained agents, one for reasoning and one for search, improves accuracy on multi-hop claim verification benchmarks.
-
Generalizable LLM Learning of Graph Synthetic Data with Post-training Alignment
Post-training alignment on synthetic graph data yields statistically significant gains on some real-world implicit-graph tasks, but gains are inconsistent and multi-step reasoning remains fragile.
-
Enhancing Social Intelligence in LLMs with Hierarchical Reasoning and Utterance-Level Goal Rewarding
LHRL-VGR, a two-stage RL method with variance-based reward routing, lets a Qwen2.5-7B agent beat GPT-4o by about 7% on SOTOPIA goal completion.
-
Design First, Code Later: Aesthetically Pleasing Template-Free Slides Generation
A design-then-code multi-agent workflow, dataset, and small trained models produce template-free slides that humans prefer over strong baselines.
-
R$^2$PO: Decoupling Rollout and Inference Policies for LLM Reasoning
R²PO trains LLM reasoning policies with a separate residual rollout head, reporting 3.4% higher MATH-500 and 1.3% higher APPS accuracy than GRPO.
-
How to Train a Leader: Hierarchical Reasoning in Multi-Agent LLMs
A leader LLM trained with a GRPO variant that conditions on frozen agent responses improves both collaborative and zero-shot accuracy on BBH, MATH, and MMLU.
-
The Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giants
Clustering-based routing plus self-consistency voting among ten 7B open models reportedly outranks GPT-4.1 and GPT-4.5 on average over 15 diverse benchmarks.
-
Beyond Single Models: Enhancing LLM Detection of Ambiguity in Requests through Debate
A leader-follower multi-agent debate protocol improves ambiguity detection for two of three tested LLMs, but the reported results lack error bars, a clear success metric, and contain internal numerical inconsistencies.
-
CogniSQL-R1-Zero: Lightweight Reinforced Reasoning for Efficient SQL Generation
A 7B model trained with GRPO and a sparse execution-correctness reward reaches 59.97% execution accuracy on BIRD dev, though the evaluation protocol and baseline numbers contain inconsistencies.
-
Mathematical methods of reinforcement learning
A survey unifying the operator-theoretic, probabilistic, and optimization-based mathematical structures underlying modern reinforcement learning algorithms.
Discussion (0). Continue with ORCID to comment.