REVIEW 11 cited by
Self-Taught Evaluators
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Model-based evaluation is at the heart of successful model development -- as a reward model for training, and as a replacement for human evaluation. To train such evaluators, the standard approach is to collect a large amount of human preference judgments over model responses, which is costly and the data becomes stale as models improve. In this work, we present an approach that aims to im-prove evaluators without human annotations, using synthetic training data only. Starting from unlabeled instructions, our iterative self-improvement scheme generates contrasting model outputs and trains an LLM-as-a-Judge to produce reasoning traces and final judgments, repeating this training at each new iteration using the improved predictions. Without any labeled preference data, our Self-Taught Evaluator can improve a strong LLM (Llama3-70B-Instruct) from 75.4 to 88.3 (88.7 with majority vote) on RewardBench. This outperforms commonly used LLM judges such as GPT-4 and matches the performance of the top-performing reward models trained with labeled examples.
Forward citations
Cited by 11 Pith papers
-
CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
A 5,352-pair benchmark shows thinking LLMs outperform non-thinking ones as code judges, but position bias and formatting sensitivity remain strong.
-
Improving Large Vision and Language Models by Learning from a Panel of Peers
A panel of LVLMs that generate, evaluate, and learn from each other's outputs improves average benchmark scores by 9 points across 15 tasks.
-
Multilingual Self-Taught Faithfulness Evaluators
STEMF trains multilingual faithfulness evaluators from synthetic data alone, and English-only training yields the best average results across languages.
-
Improving Data and Parameter Efficiency of Neural Language Models Using Representation Analysis
Representation smoothness can be used to regularize training, stop early without validation labels, and guide active learning combined with parameter-efficient fine-tuning, reducing data and compute.
-
From General to Targeted Rewards: Surpassing GPT-4 in Open-Ended Long-Context Generation
ProxyReward trains long-form generation models by rewarding how well an AI judge can answer generated yes/no questions about the response, improving open-source models on ProxyQA.
-
Beyond Gold Standards: Epistemic Ensemble of LLM Judges for Formal Mathematical Reasoning
A taxonomy-guided ensemble of LLM judges correlates with human ratings of autoformalizations (up to 0.662 on Isabelle/HOL) better than coarse-grained judges and reference metrics, but validation is partly in-sample an...
-
SkillVerse : Assessing and Enhancing LLMs with Tree Evaluation
SkillVerse clusters LLM critiques into a dendrogram of skills and uses it to improve in-context learning and predict unseen model weaknesses.
-
Learning to Reason via Mixture-of-Thought for Logical Reasoning
Jointly training and voting across natural language, code, and truth-table reasoning modalities improves LLM logical reasoning accuracy by up to 11.7 percentage points.
-
Codifying the Judge: Scalable Evaluation via Program Distillation
LLM judge logic can be distilled into a committee of Python scoring programs that match mid-size LLM judge accuracy on filtered preference datasets, at orders-of-magnitude higher throughput.
-
Advancing Multimodal Judge Models through a Capability-Oriented Benchmark and MCTS-Driven Data Generation
A capability-oriented multimodal judge benchmark and MCTS-based preference-data generation improve judge models on some benchmarks, but the paper's SOTA claim is not supported by its own numbers.
-
Reward Reasoning Model
Reward models that think before judging, trained via reinforcement learning without human-written reasoning traces, outperform standard reward models and improve with more test-time compute.
Discussion (0). Continue with ORCID to comment.