REVIEW 8 cited by
Confidence Matters: Revisiting Intrinsic Self-Correction Capabilities of Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The recent success of Large Language Models (LLMs) has catalyzed an increasing interest in their self-correction capabilities. This paper presents a comprehensive investigation into the intrinsic self-correction of LLMs, attempting to address the ongoing debate about its feasibility. Our research has identified an important latent factor - the "confidence" of LLMs - during the self-correction process. Overlooking this factor may cause the models to over-criticize themselves, resulting in unreliable conclusions regarding the efficacy of self-correction. We have experimentally observed that LLMs possess the capability to understand the "confidence" in their own responses. It motivates us to develop an "If-or-Else" (IoE) prompting framework, designed to guide LLMs in assessing their own "confidence", facilitating intrinsic self-corrections. We conduct extensive experiments and demonstrate that our IoE-based Prompt can achieve a consistent improvement regarding the accuracy of self-corrected responses over the initial answers. Our study not only sheds light on the underlying factors affecting self-correction in LLMs, but also introduces a practical framework that utilizes the IoE prompting principle to efficiently improve self-correction capabilities with "confidence". The code is available at https://github.com/MBZUAI-CLeaR/IoE-Prompting.git.
Forward citations
Cited by 8 Pith papers
-
The Computational Basis of Confidence in Large Language Models
Answer-logit differences in multimodal LMs behave as monotonic readouts of a latent decision variable in simple perceptual and memory tasks, but not in complex visual reasoning.
-
When to Vote, When to Rewrite: Disagreement-Guided Strategy Routing for Test-Time Scaling
A disagreement-guided routing framework dynamically selects among resolution, voting, and rewriting strategies for test-time scaling, delivering 3-7% accuracy gains with lower sampling cost on mathematical benchmarks.
-
MMRefine: Unveiling the Obstacles to Robust Refinement in Multimodal Large Language Models
MMRefine introduces a six-scenario, six-error-type benchmark for multimodal math refinement, and its evaluation of 17 models shows open-source models largely lag closed ones, with spatial reasoning errors the hardest to fix.
-
Spectral Origins of the Self-Correction Blind Spot in Autoregressive Generation
Self-correction blind spots in residual-stream autoregressive models arise iff the product of attention Jacobians has spectral radius ≥1, with a sharp marker threshold and RL coupling condition derived from that radius.
-
DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding
DC-Leap accelerates diffusion LLM decoding by verifying contiguous token spans at a lower confidence threshold and using high-confidence future drafts as look-ahead context, achieving up to 53x speedup with comparable...
-
Not All Errors Are Created Equal: ASCoT Addresses Late-Stage Fragility in Efficient LLM Reasoning
ASCoT claims later reasoning errors are more harmful than early ones and uses a position-weighted verifier to prune and correct CoT steps, but its key evidence is internally inconsistent.
-
Transport properties of baryon rich back-reacted thermal plasma with finite 't Hooft coupling correction
In a charged AdS black hole with Gauss-Bonnet and string-cloud corrections, drag force and jet quenching rise with GB coupling and baryon/flavor density while screening length falls; rotating-quark energy loss is supp...
-
Large Language Models for Planning: A Comprehensive and Systematic Survey
A structured survey of LLM planning methods, benchmarks, and interpretability work, organized around a three-way taxonomy.
Discussion (0). Sign in to comment.