REVIEW 6 cited by
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Prompting has shown impressive success in enabling large pretrained language models (LMs) to perform diverse NLP tasks, especially when only few downstream data are available. Automatically finding the optimal prompt for each task, however, is challenging. Most existing work resorts to tuning soft prompt (e.g., embeddings) which falls short of interpretability, reusability across LMs, and applicability when gradients are not accessible. Discrete prompt, on the other hand, is difficult to optimize, and is often created by "enumeration (e.g., paraphrasing)-then-selection" heuristics that do not explore the prompt space systematically. This paper proposes RLPrompt, an efficient discrete prompt optimization approach with reinforcement learning (RL). RLPrompt formulates a parameter-efficient policy network that generates the desired discrete prompt after training with reward. To overcome the complexity and stochasticity of reward signals by the large LM environment, we incorporate effective reward stabilization that substantially enhances the training efficiency. RLPrompt is flexibly applicable to different types of LMs, such as masked (e.g., BERT) and left-to-right models (e.g., GPTs), for both classification and generation tasks. Experiments on few-shot classification and unsupervised text style transfer show superior performance over a wide range of existing finetuning or prompting methods. Interestingly, the resulting optimized prompts are often ungrammatical gibberish text; and surprisingly, those gibberish prompts are transferrable between different LMs to retain significant performance, indicating LM prompting may not follow human language patterns.
Forward citations
Cited by 6 Pith papers
-
TRPrompt: Bootstrapping Query-Aware Prompt Optimization from Textual Rewards
TRPrompt trains an 8B prompt model directly on natural-language textual rewards and reports the highest accuracies on GSMHard and MATH among the compared methods.
-
DEMONSTRATE: Zero-shot Language to Robotic Control via Multi-task Demonstration Learning
DEMONSTRATE learns a zero-shot mapping from natural-language embeddings to MPC cost parameters from demonstrations, achieving tabletop manipulation success rates comparable to prior LLM-based pipelines.
-
Fast or Slow? Integrating Fast Intuition and Deliberate Thinking for Enhancing Visual Question Answering
FOCUS improves VQA accuracy by routing easy questions through fast zero-shot answering and hard questions through question-conditioned image segmentation before the final answer.
-
Enhancing Reasoning Capabilities of Small Language Models with Blueprints and Prompt Template Search
Small language models achieve higher accuracy on math, coding, and logic benchmarks when their prompts contain LLM-generated reasoning blueprints and a per-model, per-task searched template.
-
Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks
Simple prompt-injection attacks against real-world LLM agents like Anthropic Computer Use, MultiOn, and ChemCrow succeed at leaking data and enabling harmful actions.
-
SI-Agent: An Agentic Framework for Feedback-Driven Generation and Tuning of Human-Readable System Instructions for Large Language Models
The paper proposes a multi-agent loop (instructor, follower, feedback) to auto-generate human-readable system prompts, claiming good benchmark performance and readability, but the supporting experiments are not reprod...
Discussion (0). Continue with ORCID to comment.