REVIEW 10 cited by
Generating Sequences by Learning to Self-Correct
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Sequence generation applications require satisfying semantic constraints, such as ensuring that programs are correct, using certain keywords, or avoiding undesirable content. Language models, whether fine-tuned or prompted with few-shot demonstrations, frequently violate these constraints, and lack a mechanism to iteratively revise their outputs. Moreover, some powerful language models are of extreme scale or inaccessible, making it inefficient, if not infeasible, to update their parameters for task-specific adaptation. We present Self-Correction, an approach that decouples an imperfect base generator (an off-the-shelf language model or supervised sequence-to-sequence model) from a separate corrector that learns to iteratively correct imperfect generations. To train the corrector, we propose an online training procedure that can use either scalar or natural language feedback on intermediate imperfect generations. We show that Self-Correction improves upon the base generator in three diverse generation tasks - mathematical program synthesis, lexically-constrained generation, and toxicity control - even when the corrector is much smaller than the base generator.
Forward citations
Cited by 10 Pith papers
-
Multi-Turn On-Policy Distillation with Prefix Replay
ReOPD offline-distills multi-turn agentic LLMs via teacher-prefix replay plus step-decay sampling, matching online OPD accuracy at ≥4× speed with zero tool calls.
-
DenoiseRL: Bootstrapping Reasoning Models to Recover from Noisy Prefixes
DenoiseRL optimizes recovery from noisy prefixes in weak-model reasoning failures to improve performance and self-correction on math and general reasoning benchmarks without external supervision.
-
SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning
An RL framework that trains vision-language models to self-correct captions via a scene-graph-based reward outperforms SFT and DPO on caption quality.
-
Refining Over Resampling: Test-Time Self-Correction for LLM Reasoning
Sampling multiple reasoning paths, refining each with self-critique and self-correction, then majority voting improves math reasoning accuracy over width-only or verifier-based test-time scaling on several open-weight LLMs.
-
ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoning
ReflectRL repurposes failed expert reasoning traces as reflective scaffolding during RL and distillation training, then transitions the policy to direct reasoning, improving math and science benchmark scores.
-
I2CR: Intra- and Inter-modal Collaborative Reflections for Multimodal Entity Linking
A text-first, multi-round visual feedback framework reports state-of-the-art top-1 accuracy on WikiMEL, WikiDiverse, and RichMEL.
-
R4ec: A Reasoning, Reflection, and Refinement Framework for Recommendation Systems
R4ec trains a small reflection model to critique and refine LLM-generated user and item knowledge, which then improves downstream recommendation accuracy.
-
Hallucination Detection with Small Language Models
A multi-small-model ensemble with sentence splitting, z-score normalization, and harmonic mean detects hallucinations in RAG answers with a reported 10% F1 gain over single-model baselines.
-
Boosting LLM Reasoning via Spontaneous Self-Correction
SPOC trains LLMs to interleave self-verification and solution attempts in a single pass, reporting gains on math benchmarks, though most gains come from stronger first attempts.
-
ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflection
ReflectEvo shows that small language models can improve their reasoning by fine-tuning on their own self-generated reflections, but the headline BIG-bench gains depend on oracle feedback and an unclear data split.
Discussion (0). Continue with ORCID to comment.