REVIEW 4 cited by
Shepherd: A Critic for Language Model Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
As large language models improve, there is increasing interest in techniques that leverage these models' capabilities to refine their own outputs. In this work, we introduce Shepherd, a language model specifically tuned to critique responses and suggest refinements, extending beyond the capabilities of an untuned model to identify diverse errors and provide suggestions to remedy them. At the core of our approach is a high quality feedback dataset, which we curate from community feedback and human annotations. Even though Shepherd is small (7B parameters), its critiques are either equivalent or preferred to those from established models including ChatGPT. Using GPT-4 for evaluation, Shepherd reaches an average win-rate of 53-87% compared to competitive alternatives. In human evaluation, Shepherd strictly outperforms other models and on average closely ties with ChatGPT.
Forward citations
Cited by 4 Pith papers
-
Understanding Human Limits in Pattern Recognition: A Computational Model of Sequential Reasoning in Rock, Paper, Scissors
An LLM agent reproduces human rock-paper-scissors pattern learning, and interventions suggest that hypothesis generation, not evaluation, is the main cognitive bottleneck.
-
SkillVerse : Assessing and Enhancing LLMs with Tree Evaluation
SkillVerse clusters LLM critiques into a dendrogram of skills and uses it to improve in-context learning and predict unseen model weaknesses.
-
Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback
Training a small multi-task reward-shaping network and adding it to the RLHF reward is claimed to improve LLaMA-3-8B alignment across four benchmarks, but the supporting theory is not established.
-
Exchange of Perspective Prompting Enhances Reasoning in Large Language Models
A two-branch prompting method that exchanges answers between an original math question and a paraphrased version improves accuracy on several math benchmarks, but the gain is not separated from the extra compute or ru...
Discussion (0). Sign in to comment.