REVIEW 16 cited by
SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The recent progress in large language models (LLMs), especially the invention of chain-of-thought prompting, has made it possible to automatically answer questions by stepwise reasoning. However, when faced with more complicated problems that require non-linear thinking, even the strongest LLMs make mistakes. To address this, we explore whether LLMs are able to recognize errors in their own step-by-step reasoning, without resorting to external resources. To this end, we propose SelfCheck, a general-purpose zero-shot verification schema for recognizing such errors. We then use the results of these checks to improve question-answering performance by conducting weighted voting on multiple solutions to the question. We test SelfCheck on three datasets (GSM8K, MathQA, and MATH) and find that it successfully recognizes errors and, in turn, increases final answer accuracies.
Forward citations
Cited by 16 Pith papers
-
MentalThink: Shaping Thoughts in Mental SVG World
MLLMs that generate and render SVG sketches as multi-turn intermediate reasoning steps reach 55.1% on VSIBench and 76.0% on MindCube, far above the Qwen2.5-VL-7B backbone.
-
Beyond Semantic Equivalence: Logical Graphs for LLM Uncertainty Quantification
LGU models implication and incompatibility among LLM answers and reports consistent AUROC/AUARC gains over semantic entropy on QA benchmarks.
-
Using LLMs for Explainable, Data-Driven Insight Generation from Time Series
A modular LLM pipeline extracts explanatory factors from analyst text, conditions evidence-based report generation, and evaluates readability, consistency, and persuasiveness, reportedly matching analyst reports on re...
-
Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs
Reward-guided optimization of SAE latents, gated by reward and confidence, improves multi-benchmark reasoning and is post-hoc associated with more verification and course correction.
-
MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detection
MIND uses unlabeled similar memes, bidirectional AI insight derivation, and multi-agent debate to improve zero-shot harmful meme detection on HarM, FHM, and MAMI.
-
Leveraging Large Language Models for Tacit Knowledge Discovery in Organizational Contexts
An LLM agent can reconstruct full descriptions of data tables by conversing with simulated employees, even when the only full-knowledge expert is never contacted.
-
AdapThink: Adaptive Thinking Preferences for Reasoning Language Model
AdapThink is an RL post-training framework that adaptively reduces overthinking and underthinking in reasoning language models by rewarding confidence-appropriate reasoning depth and diverse training samples.
-
Your Agent Can Defend Itself against Backdoor Attacks
A two-level consistency defense detects backdoored LLM agents by matching thoughts to actions and reconstructed instructions to the user's instruction, reducing attack success rates on tested tasks.
-
Direct Behavior Optimization: Unlocking the Potential of Lightweight LLMs
A gradient-free Monte Carlo tree search over JSON key-step plans produces few-shot demonstrations that let LLaMA3-8B and LLaMA3.2-3B outperform GPT-3.5 on most of seven BIG-Bench Hard tasks.
-
Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing
Knowledge probing methods for LLMs are found to be highly inconsistent under minor prompt perturbations and across different probes.
-
Hume: Introducing System-2 Thinking in Visual-Language-Action Model
A dual-system vision-language-action model that improves robot control by ranking multiple sampled action chunks with a learned value function before fast execution.
-
Automatic Failure Attribution and Critical Step Prediction Method for Multi-Agent Systems Based on Causal Inference
A hierarchical causal attribution framework (performance causal inversion, Shapley values, CDC-MAS step discovery) localizes failure agents and steps in LLM multi-agent systems, reporting up to 36.2% step accuracy.
-
I2CR: Intra- and Inter-modal Collaborative Reflections for Multimodal Entity Linking
A text-first, multi-round visual feedback framework reports state-of-the-art top-1 accuracy on WikiMEL, WikiDiverse, and RichMEL.
-
InfoFlood: Jailbreaking Large Language Models with Information Overload
InfoFlood claims near-perfect jailbreak success on four frontier LLMs by rewriting harmful queries into verbose academic prose with fake citations, past-tense framing, and ethical disclaimers, without adversarial suffixes.
-
Generating on Generated: An Approach Towards Self-Evolving Diffusion Models
A recursive self-training loop that filters prompts, selects preferred images, and reweights out-of-distribution samples improves Stable Diffusion models over multiple rounds.
-
Beyond ROUGE: N-Gram Subspace Features for LLM Hallucination Detection
Singular values of label-grouped n-gram frequency tensors are used as MLP features for hallucination detection, with reported gains on HaluEval that rely on label-aware grouping.
Discussion (0). Sign in to comment.