REVIEW 8 cited by
Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We present Step-Back Prompting, a simple prompting technique that enables LLMs to do abstractions to derive high-level concepts and first principles from instances containing specific details. Using the concepts and principles to guide reasoning, LLMs significantly improve their abilities in following a correct reasoning path towards the solution. We conduct experiments of Step-Back Prompting with PaLM-2L, GPT-4 and Llama2-70B models, and observe substantial performance gains on various challenging reasoning-intensive tasks including STEM, Knowledge QA, and Multi-Hop Reasoning. For instance, Step-Back Prompting improves PaLM-2L performance on MMLU (Physics and Chemistry) by 7% and 11% respectively, TimeQA by 27%, and MuSiQue by 7%.
Forward citations
Cited by 8 Pith papers
-
Training Language Models to Cooperate with Inference-Time Controllers
Multi-controller post-training with turn-level GRPO over reusable reasoning modules improves LLM transfer to held-out and shifted inference-time controllers versus single-controller RL.
-
Orchestration for Domain-specific Edge-Cloud Language Models
ECO-LLM jointly selects query processing, retrieval, and model components per query, cutting cost by 60% and latency up to 6x versus model routing in edge-cloud tests.
-
Which Prompting Technique Should I Use? An Empirical Investigation of Prompting Techniques for Software Engineering Tasks
Across ten software engineering tasks and four LLMs, no prompting technique wins consistently; ES-KNN is best on many tasks, some techniques underperform the baseline, and USC is best for code QA and code generation.
-
Vis-CoT: A Human-in-the-Loop Framework for Interactive Visualization and Intervention in LLM Chain-of-Thought Reasoning
An interactive chain-of-thought editor with prune and graft interventions reports large accuracy gains, but without error bars, code, or a comparison to existing editing tools.
-
Prompt Engineering for Requirements Engineering: A Literature Review and Roadmap
The first roadmap-oriented systematic literature review of prompt engineering for requirements engineering analyzes 35 studies and proposes a hybrid taxonomy and research roadmap.
-
PromptGuard: An Orchestrated Prompting Framework for Principled Synthetic Text Generation for Vulnerable Populations using LLMs with Enhanced Safety, Fairness, and Controllability
A framework paper that claims its VulnGuard prompt technique cuts harmful LLM outputs by 25-30% via theoretical bounds, without a real proof or empirical test.
-
Mixture of Reasonings: Teach Large Language Models to Reason with Adaptive Strategies
MoR fine-tunes Qwen2.5 on GPT-4o-selected reasoning templates, claiming up to 13.5% accuracy gains, but the reported gains are not robustly supported.
-
Exp4Fuse: A Rank Fusion Framework for Enhanced Sparse Retrieval using Large Language Model-based Query Expansion
Exp4Fuse improves sparse retrieval by fusing the ranked lists from the original query and an LLM-expanded query using a modified reciprocal rank fusion.
Discussion (0). Continue with ORCID to comment.