REVIEW 19 cited by
V-STaR: Training Verifiers for Self-Taught Reasoners
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Common self-improvement approaches for large language models (LLMs), such as STaR, iteratively fine-tune LLMs on self-generated solutions to improve their problem-solving ability. However, these approaches discard the large amounts of incorrect solutions generated during this process, potentially neglecting valuable information in such solutions. To address this shortcoming, we propose V-STaR that utilizes both the correct and incorrect solutions generated during the self-improvement process to train a verifier using DPO that judges correctness of model-generated solutions. This verifier is used at inference time to select one solution among many candidate solutions. Running V-STaR for multiple iterations results in progressively better reasoners and verifiers, delivering a 4% to 17% test accuracy improvement over existing self-improvement and verification approaches on common code generation and math reasoning benchmarks with LLaMA2 models.
Forward citations
Cited by 19 Pith papers
-
The Verifier is the Curriculum: Execution-Gated Self-Distillation for Cross-Family Game Generation
Strict-launch-gated rejection-sampling self-distillation compounds cross-family clean Godot generation from 8.8% to 42.2% and full best-of-K coverage, while gold duplication and a lenient BUILD filter erase the gain.
-
Cybersecurity Detection Classification with Reasoning-enabled Language Models
CoT-trained triage plus a separate reasoning calibrator reaches 82.6% accuracy and large high-confidence recall gains over direct-label LLM classifiers on real SOC detections.
-
LeAct: Learning to Reason from Expert Actions
An AI can learn to reason by sampling explanations for an expert's actions and keeping only the ones that help it predict those actions.
-
Test-Time Scaling via Error Localization
TTEL uses feedback-induced token probability drops to localize the first error in a failed reasoning trace and branch a new generation from that prefix, improving pass@k per token on coding and math benchmarks.
-
Post-Training Shifts Confidence: A Three-Stage Analysis of How SFT, RL, and OPD Shape CoT Calibration
Post-training changes where confidence is trustworthy in reasoning traces: OPD best gauges difficulty before thinking, SFT best supports early stopping, RL best selects final answers, and confidence should be read onl...
-
LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models
A confidence-based when-to-look gate plus semantic-guided attention where-to-look module improves training-free fine-grained visual reasoning accuracy and reduces inference cost versus search-based baselines.
-
CAViAR: Critic-Augmented Video Agentic Reasoning
CAViAR, an agent-plus-critic system for long video reasoning, improves on direct video LLM inference across LVBench, Neptune, and ActivityNet-RTL.
-
Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models
TraceRL aligns the RL objective with the DLM's actual step-by-step decoding, producing TraDo-4B/8B models that beat autoregressive baselines on math reasoning.
-
Learn from What We HAVE: History-Aware VErifier that Reasons about Past Interactions Online
A history-aware verifier that scores candidate actions using past interactions cuts failure rates in ambiguous robot manipulation tasks compared to using the generator alone.
-
VL-GenRM: Enhancing Vision-Language Verification via Vision Experts and Iterative Training
A vision-expert-filtered, chain-of-thought-guided, iteratively fine-tuned reward model boosts a compact 7B model's ability to judge vision-language responses, especially detecting hallucinations.
-
Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification
An iterative, verifier-guided visual token scaling framework improves multi-step visual reasoning in both closed and open multimodal models on BLINK and related benchmarks.
-
Sample More, Reflect Less: Self-Refine and Reflexion Lose to Repeated Sampling at Equal Token Cost, from 1.5B to 7B
Self-inspection methods (Self-Refine, Reflexion, Best-of-N self-verify) lose to equal-token repeated sampling on math from 1.5B to 7B; no tested method reliably wins.
-
Reasoning in machine vision by learning fast and slow thinking
A dual-process vision system improves segmentation accuracy by spending more inference-time compute, using a fast predictor and a slow self-play refiner, reporting gains on cancer localisation with only 8-16 labels.
-
See, Think, Learn: A Self-Taught Multimodal Reasoner
A self-training loop that structures model rationales into image-caption, reasoning, and conclusion, and adds negative rationales, improves VLM accuracy on M3CoT over answer-only and STaR baselines.
-
TaoSR-AGRL: Adaptive Guided Reinforcement Learning Framework for E-commerce Search Relevance
TaoSR-AGRL improves e-commerce search relevance by combining dense rule-aware reward shaping with adaptive ground-truth-guided replay in GRPO, reporting gains on Taobao's private offline and online evaluations.
-
Logit Arithmetic Elicits Long Reasoning Capabilities Without Training
ThinkLogit blends logits from a small reasoning guider into a frozen 32B model, improving math pass@1 by up to 29% without training the large model.
-
DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoning
Separating text and icon grounding with iterative zooming improves GUI-element localization accuracy of existing vision-language models without retraining.
-
The Hitchhiker's Guide to Agentic AI: From Foundations to Systems
A survey-style reference book mapping the full agentic-AI stack from transformer internals to production deployment, with no new research result.
-
A Survey on Large Language Models for Mathematical Reasoning
Recent advances in LLM mathematical reasoning are organized into comprehension and generation phases, covering methods from prompting to test-time scaling.
Discussion (0). Sign in to comment.