REVIEW 12 cited by
Reinforcement Learning Enhanced LLMs: A Survey
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Reinforcement learning (RL) enhanced large language models (LLMs), particularly exemplified by DeepSeek-R1, have exhibited outstanding performance. Despite the effectiveness in improving LLM capabilities, its implementation remains highly complex, requiring complex algorithms, reward modeling strategies, and optimization techniques. This complexity poses challenges for researchers and practitioners in developing a systematic understanding of RL-enhanced LLMs. Moreover, the absence of a comprehensive survey summarizing existing research on RL-enhanced LLMs has limited progress in this domain, hindering further advancements. In this work, we are going to make a systematic review of the most up-to-date state of knowledge on RL-enhanced LLMs, attempting to consolidate and analyze the rapidly growing research in this field, helping researchers understand the current challenges and advancements. Specifically, we (1) detail the basics of RL; (2) introduce popular RL-enhanced LLMs; (3) review researches on two widely-used reward model-based RL techniques: Reinforcement Learning from Human Feedback (RLHF) and Reinforcement Learning from AI Feedback (RLAIF); and (4) explore Direct Preference Optimization (DPO), a set of methods that bypass the reward model to directly use human preference data for aligning LLM outputs with human expectations. We will also point out current challenges and deficiencies of existing methods and suggest some avenues for further improvements. Project page of this work can be found at https://github.com/ShuheWang1998/Reinforcement-Learning-Enhanced-LLMs-A-Survey.
Forward citations
Cited by 12 Pith papers
-
AR$^2$: Adversarial Reinforcement Learning for Abstract Reasoning in Large Language Models
Adversarial teacher-student RL on computationally equivalent narrative rewrites of coding problems yields modest but consistent pass@1 gains on competitive programming benchmarks.
-
CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning
A contrastive reinforcement learning framework, CUDA-L1, trains an LLM to optimize CUDA kernels and reports 3.12x average speedup on KernelBench, though evaluation is on the same tasks used for training.
-
Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities
A survey that groups graph-empowered AI agent research into planning, execution, memory, and multi-agent coordination, plus agents-for-graphs and applications.
-
From Emergence to Control: Probing and Modulating Self-Reflection in Language Models
Self-reflection in LLMs can be steered up or down by a single activation-space vector, improving accuracy when amplified and cutting output length when suppressed.
-
Three Minds, One Legend: Jailbreak Large Reasoning Model with Adaptive Stacked Ciphers
A stacked-cipher jailbreak with adaptive code selection achieves 80.8% to 100% attack success on commercial large reasoning models.
-
Accelerating Reinforcement Learning Algorithms Convergence using Pre-trained Large Language Models as Tutors With Advice Reusing
LLM tutoring modestly accelerates RL convergence on average, with advice reuse saving wall-clock time but reducing stability.
-
Prompt Informed Reinforcement Learning for Visual Coverage Path Planning
Adding GPT-3.5 semantic recommendations as an auxiliary reward term to PPO improves visual coverage and reduces redundancy for simulated aerial coverage path planning, according to reported experiments.
-
UrbanMind: Towards Urban General Intelligence via Tool-Enhanced Retrieval-Augmented Generation and Multilevel Optimization
The paper introduces UrbanMind, a tool-enhanced RAG framework with a multilevel optimization formulation for continual adaptation in urban AI, but offers only qualitative prototype results.
-
Efficient Online RFT with Plug-and-Play LLM Judges: Unlocking State-of-the-Art Performance
A frozen 7B-8B LLM with a JSON rubric and a small LoRA adapter is claimed to outperform 27B-70B reward models and enable 92% GSM-8K exact match under online PPO.
-
Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
A survey that maps reinforcement learning methods, datasets, benchmarks, and open-source tools across the full training lifecycle of large language models, focusing on verifiable-reward reasoning.
-
A Technical Survey of Reinforcement Learning Techniques for Large Language Models
A survey of RL methods for LLMs that organizes the field by reward modeling, feedback source, and optimization strategy, with benchmark tables favoring a scalar-regression UNA variant over DPO and KTO in offline alignment.
-
RACE-Align: Retrieval-Augmented and Chain-of-Thought Enhanced Preference Alignment for Large Language Models
RACE-Align generates preference pairs from RAG-grounded chain-of-thought answers and applies DPO to align a 1.7B model, showing improved reasoning scores in TCM QA but lacking statistical support.
Discussion (0). Continue with ORCID to comment.