REVIEW 7 cited by
MARIO: MAth Reasoning with code Interpreter Output -- A Reproducible Pipeline
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have seen considerable advancements in natural language understanding tasks, yet there remains a gap to bridge before attaining true artificial general intelligence, especially concerning shortcomings in mathematical reasoning capabilities. We postulate that the inherent nature of LLM training, which focuses on predicting probabilities of next token, presents challenges in effectively modeling mathematical reasoning that demands exact calculations, both from data-driven and theoretical standpoints. In this paper, we address this challenge by enriching the data landscape and introducing a novel math dataset, enhanced with a capability to utilize a Python code interpreter. This dataset is derived from GSM8K and MATH and has been further refined through a combination of GPT-4 annotations, human review, and self-training processes, where the errors in the original GSM8K training set have been fixed. Additionally, we propose a tentative, easily replicable protocol for the fine-tuning of math-specific LLMs, which has led to a significant improvement in the performance of a 7B-parameter LLM on the GSM8K and MATH datasets. We are committed to advancing the field of mathematical reasoning in LLMs and, to that end, we have made source code for data generation / training / inference, and the model checkpoints publicly available at \url{https://github.com/MARIO-Math-Reasoning/MARIO}. We hope this will facilitate further research and development within the community.
Forward citations
Cited by 7 Pith papers
-
SCOPE: Compress Mathematical Reasoning Steps for Efficient Automated Process Annotation
SCOPE compresses sampled math reasoning traces into a prefix tree of AST-normalized code steps, yielding PRM training labels at O(N) cost and improving Best-of-N and ProcessBench scores over automated baselines.
-
Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning
A 3B agent trained with supervised tool-use traces and reinforcement learning answers week-long egocentric video questions by dynamically selecting hierarchical retrieval, video-LLM, and VLM tools.
-
Rectified Sparse Attention
ReSA keeps sparse decoding accurate by refreshing the KV cache with dense attention every f tokens, achieving near-lossless quality with a large speedup.
-
Visual Agentic Reinforcement Fine-Tuning
Reinforcement fine-tuning with verifiable rewards enables open-source vision-language models to use web search and image-processing code, improving visual QA and multi-hop reasoning.
-
Running in CIRCLE? A Simple Benchmark for LLM Code Interpreter Security
CIRCLE is a 1,260-prompt benchmark that measures how often commercial LLM code interpreters refuse, execute, or time out on resource-exhaustion tasks, revealing large and inconsistent safety gaps.
-
GSM-Plus-BN: A Perturbation-Based Benchmark for Bangla Mathematical Reasoning in Large Language Models
The paper releases GSM-Plus-BN, a human-verified Bengali translation of the GSM-Plus perturbed math benchmark, and reports accuracy baselines for six open LLMs under standard and CoT prompting.
-
Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey
A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.
Discussion (0). Continue with ORCID to comment.