REVIEW 15 cited by
Learning To Retrieve Prompts for In-Context Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In-context learning is a recent paradigm in natural language understanding, where a large pre-trained language model (LM) observes a test instance and a few training examples as its input, and directly decodes the output without any update to its parameters. However, performance has been shown to strongly depend on the selected training examples (termed prompt). In this work, we propose an efficient method for retrieving prompts for in-context learning using annotated data and a LM. Given an input-output pair, we estimate the probability of the output given the input and a candidate training example as the prompt, and label training examples as positive or negative based on this probability. We then train an efficient dense retriever from this data, which is used to retrieve training examples as prompts at test time. We evaluate our approach on three sequence-to-sequence tasks where language utterances are mapped to meaning representations, and find that it substantially outperforms prior work and multiple baselines across the board.
Forward citations
Cited by 15 Pith papers
-
MicLog: Towards Accurate and Efficient LLM-based Log Parsing via Progressive Meta In-Context Learning
A small open-source LLM trained with progressive meta in-context learning parses logs more accurately and faster than prior state-of-the-art LLM parsers on Loghub-2.0.
-
Context Engineering for Trustworthiness: Rescorla Wagner Steering Under Mixed and Inappropriate Contexts
LLMs amplify the least-dominant context, so one bad sentence among twenty good ones distorts answers; joint judgment-and-answer fine-tuning (RW-Steering) stabilizes response quality across contamination levels from 0% to 95%.
-
Failures Are the Stepping Stones to Success: Enhancing Few-Shot In-Context Learning by Leveraging Negative Samples
A retrieval method that uses semantically similar wrong answers as anchors to find corrective correct demonstrations improves few-shot in-context learning accuracy on seven reasoning datasets.
-
Maximally-Informative Retrieval for State Space Model Generation
RICO ranks documents by how much they reduce an SSM's question perplexity, using gradient-document inner products, and matches BM25 while often beating E5 on answer quality without finetuning.
-
Mobile Traffic Prediction using LLMs with Efficient In-context Demonstration Selection
A two-step in-context demonstration selection method, based on distance similarity and zero-shot prediction error, improves LLM-based mobile traffic prediction on a real 5G dataset compared with zero-shot and simple b...
-
ConText: Driving In-context Learning for Text Removal and Segmentation
ConText is the first visual in-context learning model for text removal and segmentation, chaining the two tasks and using self-prompting to reach new state-of-the-art scores.
-
Learning to Select In-Context Demonstration Preferred by Large Language Model
A generative preference-learning method trains a latent demonstration selector from LLM feedback and improves few-shot in-context learning performance on most of 19 benchmark datasets.
-
DCG-SQL: Enhancing In-Context Learning for Text-to-SQL with Deep Contextual Schema Link Graph
DCG-SQL retrieves text-to-SQL demonstrations by embedding a question-to-schema link graph, improving execution accuracy on Spider by up to about 10 points over random demonstrations on small LLMs.
-
Learning to Select Visual In-Context Demonstrations
A Dueling-DQN agent selects visual in-context demonstrations and outperforms kNN retrieval on objective regression benchmarks but not on subjective preference tasks, per the paper's main table.
-
InSQuAD: In-Context Learning for Efficient Retrieval via Submodular Mutual Information to Enforce Quality and Diversity
A submodular mutual information framework for selecting and training in-context learning exemplars improves average accuracy on nine benchmarks by about five points over the IDEAL baseline.
-
ICM-Fusion: In-Context Meta-Optimized LoRA Fusion for Multi-Task Adaptation
ICM-Fusion uses a conditional VAE plus task-vector guidance to fuse multiple LoRA adapters into one model, reporting marginal average gains on vision and language benchmarks and larger gains in a few-shot long-tail setup.
-
DICE: Dynamic In-Context Example Selection in LLM Agents via Efficient Knowledge Transfer
DICE dynamically retrieves the most relevant in-context demonstrations at each agent step, and in this preprint it raises exact-match and success-rate scores on HotpotQA, ALFWorld, and Webshop across ReAct, Reflexion,...
-
Prompt Engineering for Requirements Engineering: A Literature Review and Roadmap
The first roadmap-oriented systematic literature review of prompt engineering for requirements engineering analyzes 35 studies and proposes a hybrid taxonomy and research roadmap.
-
Unveiling Effective In-Context Configurations for Image Captioning: An External & Internal Analysis
For Flamingo-style models, increasing the number of in-context examples improves language coherence but degrades visual-text alignment, and similarity-based image retrieval inflates CIDEr scores by encouraging caption...
-
Control-R: Towards controllable test-time scaling
Control-R-32B is a fine-tuned 32B model that takes 11 control-field scores in the prompt, claiming to let users trade reasoning depth for efficiency and reaching SOTA on AIME2024 and MATH500 at that scale.
Discussion (0). Continue with ORCID to comment.