REVIEW 10 cited by
Can language models learn from explanations in context?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Language Models (LMs) can perform new tasks by adapting to a few in-context examples. For humans, explanations that connect examples to task principles can improve learning. We therefore investigate whether explanations of few-shot examples can help LMs. We annotate questions from 40 challenging tasks with answer explanations, and various matched control explanations. We evaluate how different types of explanations, instructions, and controls affect zero- and few-shot performance. We analyze these results using statistical multilevel modeling techniques that account for the nested dependencies among conditions, tasks, prompts, and models. We find that explanations can improve performance -- even without tuning. Furthermore, explanations hand-tuned for performance on a small validation set offer substantially larger benefits, and building a prompt by selecting examples and explanations together substantially improves performance over selecting examples alone. Finally, even untuned explanations outperform carefully matched controls, suggesting that the benefits are due to the link between an example and its explanation, rather than lower-level features. However, only large models benefit. In summary, explanations can support the in-context learning of large LMs on challenging tasks.
Forward citations
Cited by 10 Pith papers
-
ReGUIDE: Data Efficient GUI Grounding via Spatial Reasoning and Search
ReGUIDE reaches state-of-the-art GUI grounding accuracy using 0.2% of the usual training data by adding self-generated reasoning, spatial-consistency training, and test-time KDE coordinate search.
-
Test-Time Scaling via Error Localization
TTEL uses feedback-induced token probability drops to localize the first error in a failed reasoning trace and branch a new generation from that prefix, improving pass@k per token on coding and math benchmarks.
-
Computational models of pragmatic reasoning with flexible generation of meaning and expression alternatives
Combining language-model generation with rule-based selection reproduces several pragmatic phenomena, but the language models only worked reliably as idea generators, not as judges of formal linguistic properties.
-
Understanding and evaluating computer vision models through the lens of counterfactuals
Counterfactual-based methods for concept attribution in classifiers and for dynamic bias evaluation and mitigation in text-to-image models.
-
Can AI Examine Novelty of Patents?: Novelty Evaluation Based on the Correspondence between Patent Claim and Prior Art
A new patent novelty benchmark from real examiner rejections shows large language models can classify novelty at about 62% accuracy, while smaller classification models perform at chance.
-
Data-Efficient Adaptation of LLMs via Attention Head Reweighting
Learning a single scalar per attention head lets LLMs adapt to few-shot text classification better than LoRA, with 200–1000x fewer trainable parameters.
-
Reasoning or Overthinking: Evaluating Large Language Models on Financial Sentiment Analysis
On financial sentiment classification, zero-shot LLMs match human labels better without chain-of-thought reasoning than with it.
-
TT-XAI: Trustworthy Clinical Text Explanations via Keyword Distillation and LLM Reasoning
Keyword distillation of clinical notes improved BERT predictions and explanation quality in small tests, but effects are modest and some important details are unverified.
-
VerifyLLM: LLM-Based Pre-Execution Task Plan Verification for Robots
An LLM plus LTL-based verification module that reorders, inserts, and removes steps in household robot plans, reporting reduced ordering errors but with weak experimental support.
-
Towards Transparent AI: A Survey on Explainable Large Language Models
A review that groups LLM explainability methods by transformer architecture and discusses their evaluation and applications.
Discussion (0). Continue with ORCID to comment.