REVIEW 7 cited by
Teaching Algorithmic Reasoning via In-context Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have shown increasing in-context learning capabilities through scaling up model and data size. Despite this progress, LLMs are still unable to solve algorithmic reasoning problems. While providing a rationale with the final answer has led to further improvements in multi-step reasoning problems, Anil et al. 2022 showed that even simple algorithmic reasoning tasks such as parity are far from solved. In this work, we identify and study four key stages for successfully teaching algorithmic reasoning to LLMs: (1) formulating algorithms as skills, (2) teaching multiple skills simultaneously (skill accumulation), (3) teaching how to combine skills (skill composition) and (4) teaching how to use skills as tools. We show that it is possible to teach algorithmic reasoning to LLMs via in-context learning, which we refer to as algorithmic prompting. We evaluate our approach on a variety of arithmetic and quantitative reasoning tasks, and demonstrate significant boosts in performance over existing prompting techniques. In particular, for long parity, addition, multiplication and subtraction, we achieve an error reduction of approximately 10x, 9x, 5x and 2x respectively compared to the best available baselines.
Forward citations
Cited by 7 Pith papers
-
When LLMs Stop Following Steps: A Diagnostic Study of Procedural Execution in Language Models
LLM first-answer accuracy on procedural arithmetic drops from 61% on 5-step tasks to 20% on 95-step tasks, with frequent failures including skipped steps, premature answers, and hallucinated operations.
-
Primal-Dual Neural Algorithmic Reasoning
A GNN framework that simulates primal-dual approximation algorithms for NP-hard problems and, with small-instance optimal labels, can beat the algorithm it learns.
-
A Group Theoretic Analysis of the Symmetries Underlying Base Addition and Their Learnability by Neural Networks
For bases 3-5, the structural complexity of a base-addition carry rule (fractal dimension, carry frequency, associativity) strongly predicts whether a tiny recurrent network can learn it and generalize from 3-digit to...
-
EPiC: Towards Lossless Speedup for Reasoning Training through Edge-Preserving CoT Condensation
Keeping only the first and last parts of chain-of-thought training traces preserves math reasoning accuracy on MATH500 at half the sequence length, but not on harder benchmarks.
-
Retrieval-Augmented Generation of Ontologies from Relational Databases
An iterative RAG-LLM pipeline converts relational schemas into OWL ontology fragments, achieving LLM-judged quality scores of 4.2 to 4.6 out of 5 on two medical databases.
-
Future-KL Regularized GRPO: Process-Level Credit Assignment from $f$-Divergence Regularization
Abstract claims FRPO adds a future-KL correction to GRPO for better math reasoning; the provided manuscript body is a different PRL paper and does not support those claims.
-
A Survey on Large Language Models for Mathematical Reasoning
Recent advances in LLM mathematical reasoning are organized into comprehension and generation phases, covering methods from prompting to test-time scaling.
Discussion (0). Continue with ORCID to comment.