Pith. sign in

REVIEW 12 cited by

Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2504.05812 v3 pith:VKWBDRDN submitted 2025-04-08 cs.LG

Qingyang Zhang , Haitao Wu , Changqing Zhang , Peilin Zhao , Yatao Bian This is my paper · ORCID
classification cs.LG
keywords reasoningourssupervisedaccuracybaseentropyfullyincentivization
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Existing methods to enhance the reasoning capability of large language models predominantly rely on supervised fine-tuning (SFT) followed by reinforcement learning (RL) on reasoning-specific data. These approaches critically depend on external supervisions--such as labeled reasoning traces, verified golden answers, or pre-trained reward models. In this work, we propose Entropy Minimized Policy Optimization (\ours), which makes an early attempt at fully unsupervised LLM reasoning incentivization. By continuously minimizing the predictive entropy of LLMs on unlabeled questions in a latent semantic space, \ours achieves competitive performance compared to supervised counterparts on both mathematical and free-form natural reasoning tasks. Specifically, without any supervised signals, \ours boosts the accuracy of Qwen2.5-Math-7B Base from 30.7\% to 48.1\% on mathematical benchmarks and improves the accuracy of Qwen2.5-7B Base from 32.1\% to 50.1\% on MMLU-Pro. Primary experiments and analysis are also provided to interpret the effectiveness of \ours. Code is available at https://github.com/QingyangZhang/EMPO.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. On-Policy Self-Distillation without Any Supervision

    cs.LG 2026-08 conditional novelty 6.0 of 10

    A language model improves its own math reasoning by distilling its majority-vote consensus into the prefixes of its own disagreeing answers, with no external labels.

  2. CPMobius: Iterative Coach-Player Reasoning for Data-Free Reinforcement Learning

    cs.CL 2026-02 conditional novelty 6.0 of 10

    CPMobius uses iterative coach-player reinforcement learning to improve mathematical reasoning in LLMs without external training data, yielding +4.9 average accuracy gains on Qwen2.5-Math-7B-Instruct.

  3. Breaking the Self-Confirming Loop: Diagnosing and Mitigating Systemic Reward Bias in Self-Rewarding RL

    cs.LG 2025-10 conditional novelty 6.0 of 10

    Self-rewarding RL can be stabilized by ensembling multiple policy models' majority-vote rewards, reaching within 3.6% of verifiable-reward RL on math benchmarks.

  4. Self-Evolving Vision-Language Models for Image Quality Assessment via Voting and Ranking

    cs.CV 2025-09 conditional novelty 6.0 of 10

    EvoQuality lets a VLM self-train for image quality assessment using majority-voted pairwise preferences as pseudo-labels, improving PLCC by about 32% and rivaling supervised models.

  5. Know When to Explore: Difficulty-Aware Certainty as a Guide for LLM Reinforcement Learning

    cs.AI 2025-08 conditional novelty 6.0 of 10

    A new RL reward-shaping method that rewards low confidence on hard problems and high confidence on easy ones improves LLM math reasoning over a GRPO baseline.

  6. Maximizing Prefix-Confidence at Test-Time Efficiently Improves Mathematical Reasoning

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Selecting the most confident 32-token prefix and completing only it gives better accuracy per compute than majority voting on five math reasoning datasets, using only the model's own confidence as a selector.

  7. Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning

    cs.AI 2025-06 conditional novelty 6.0 of 10

    CoVo trains LLMs with a self-generated reward based on the consistency and volatility of intermediate reasoning states, matching supervised RL performance without external labels.

  8. Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions

    cs.CV 2026-07 accept novelty 5.0 of 10

    CTTA methods fall into optimization-based, parameter-efficient, and architecture-based families that adapt pretrained vision models online under continual unlabeled shifts while fighting forgetting and error accumulation.

  9. Plan Then Action:High-Level Planning Guidance Reinforcement Learning for LLM Reasoning

    cs.AI 2025-10 conditional novelty 5.0 of 10

    A plan-then-reason SFT plus a plan-quality reward in GRPO improves math-reasoning accuracy by small but consistent margins over GRPO and DAPO.

  10. Harnessing Uncertainty: Entropy-Modulated Policy Gradients for Long-Horizon LLM Agents

    cs.LG 2025-09 conditional novelty 5.0 of 10

    EMPG re-weights policy-gradient updates by step-level token entropy, amplifying confident correct actions and muting uncertain ones, and adds a future-clarity bonus.

  11. No Free Lunch: Rethinking Internal Feedback for LLM Reasoning

    cs.LG 2025-06 conditional novelty 5.0 of 10

    Internal feedback rewards (entropy and self-certainty) improve base LLM math reasoning only in early training and degrade later, with little benefit for instruct models.

  12. Reinforcing Video Reasoning with Focused Thinking

    cs.CV 2025-05 reject novelty 5.0 of 10

    A GRPO variant with token-level KL weighting and partial-credit rewards improves video-QA on modified multi-answer benchmarks, but transfer to original single-answer benchmarks is not established.

Pith tools