REVIEW 6 cited by
On the Brittle Foundations of ReAct Prompting for Agentic Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The reasoning abilities of Large Language Models (LLMs) remain a topic of debate. Some methods such as ReAct-based prompting, have gained popularity for claiming to enhance sequential decision-making abilities of agentic LLMs. However, it is unclear what is the source of improvement in LLM reasoning with ReAct based prompting. In this paper we examine these claims of ReAct based prompting in improving agentic LLMs for sequential decision-making. By introducing systematic variations to the input prompt we perform a sensitivity analysis along the claims of ReAct and find that the performance is minimally influenced by the "interleaving reasoning trace with action execution" or the content of the generated reasoning traces in ReAct, contrary to original claims and common usage. Instead, the performance of LLMs is driven by the similarity between input example tasks and queries, implicitly forcing the prompt designer to provide instance-specific examples which significantly increases the cognitive burden on the human. Our investigation shows that the perceived reasoning abilities of LLMs stem from the exemplar-query similarity and approximate retrieval rather than any inherent reasoning abilities.
Forward citations
Cited by 6 Pith papers
-
NeSyFS: A Neuro-symbolic Fast-Slow Thinking Framework for LLM Agent under Partial Observability
NeSyFS combines a knowledge-graph belief state, a reflection step, and TSMC-style particle planning to improve LLM agent success rates on ALFWorld, WebShop, and ScienceWorld.
-
Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language
Current LLMs produce consistent but miscalibrated natural-language descriptors of likelihood and uncertainty from probabilistic predictions and are not yet reliable zero-shot risk communicators.
-
RIMRULE: Improving Tool-Using Language Agents via MDL-Guided Rule Learning
RIMRULE distills LLM tool-use failures into MDL-compressed symbolic rules that, when injected at inference, improve tool-calling accuracy and transfer across models.
-
Iterative Deepening Sampling as Efficient Test-Time Scaling
ID-Sampling inserts self-correction trigger sentences at geometrically increasing token budgets during LLM generation, improving pass rates on MATH-500 and AIME over vanilla Best-of-N and majority voting.
-
CogniConsole: Externalizing Inference-Time Control as a Formal Abstraction for Reliable LLM Interactions
Under a fixed model, increasing structural scaffolding of inference-time control systematically reduces LLM output variance and failure rates on controllability probes.
-
Toolsuite for Implementing Multiagent Systems Based on Communication Protocols
A consolidated description of a tool suite for specifying, verifying, and implementing multiagent systems through information protocols, with no new research result beyond the authors' prior tool papers.
Discussion (0). Continue with ORCID to comment.