Pith. sign in

REVIEW 3 cited by

UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2303.08518 v4 pith:WP7HBR34 submitted 2023-03-15 cs.CL

Daixuan Cheng , Shaohan Huang , Junyu Bi , Yuefeng Zhan , Jianfeng Liu , Yujing Wang , Hao Sun , Furu Wei
show 2 more authors
Denvy Deng Qi Zhang
This is my paper
classification cs.CL
keywords retrieverllmspromptuprisezero-shotevaluationimprovingretrieval
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large Language Models (LLMs) are popular for their impressive abilities, but the need for model-specific fine-tuning or task-specific prompt engineering can hinder their generalization. We propose UPRISE (Universal Prompt Retrieval for Improving zero-Shot Evaluation), which tunes a lightweight and versatile retriever that automatically retrieves prompts for a given zero-shot task input. Specifically, we demonstrate universality in a cross-task and cross-model scenario: the retriever is tuned on a diverse set of tasks, but tested on unseen task types; we use a small frozen LLM, GPT-Neo-2.7B, for tuning the retriever, but test the retriever on different LLMs of much larger scales, such as BLOOM-7.1B, OPT-66B and GPT3-175B. Additionally, we show that UPRISE mitigates the hallucination problem in our experiments with ChatGPT, suggesting its potential to improve even the strongest LLMs. Our model and code are available at https://github.com/microsoft/LMOps.

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Retrieval-Augmented Generation as Noisy In-Context Learning: A Unified Theory and Risk Bounds

    cs.LG 2025-06 conditional novelty 7.0 of 10

    RAG in in-context linear regression has an exact bias-variance tradeoff and a finite-sample bound revealing a generalization ceiling as retrieved examples grow.

  2. Learning to Select In-Context Demonstration Preferred by Large Language Model

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A generative preference-learning method trains a latent demonstration selector from LLM feedback and improves few-shot in-context learning performance on most of 19 benchmark datasets.

  3. A Survey on Proactive Defense Strategies Against Misinformation in Large Language Models

    cs.IR 2025-07 reject novelty 3.0 of 10

    A survey claims proactive defenses against LLM misinformation outperform post-hoc detection by up to 63%, but no meta-analysis details are provided to support the claim.

Pith tools