Pith. sign in

REVIEW 17 cited by

MathPrompter: Mathematical Reasoning using Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2303.05398 v1 pith:LIR2IW5E submitted 2023-03-04 cs.CL cs.AI

Shima Imani , Liang Du , Harsh Shrivastava This is my paper
classification cs.CLcs.AI
keywords llmslanguagemathpromptermodelstechniquearithmeticconfidenceimproves
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

Large Language Models (LLMs) have limited performance when solving arithmetic reasoning tasks and often provide incorrect answers. Unlike natural language understanding, math problems typically have a single correct answer, making the task of generating accurate solutions more challenging for LLMs. To the best of our knowledge, we are not aware of any LLMs that indicate their level of confidence in their responses which fuels a trust deficit in these models impeding their adoption. To address this deficiency, we propose `MathPrompter', a technique that improves performance of LLMs on arithmetic problems along with increased reliance in the predictions. MathPrompter uses the Zero-shot chain-of-thought prompting technique to generate multiple Algebraic expressions or Python functions to solve the same math problem in different ways and thereby raise the confidence level in the output results. This is in contrast to other prompt based CoT methods, where there is no check on the validity of the intermediate steps followed. Our technique improves over state-of-the-art on the MultiArith dataset ($78.7\%\rightarrow92.5\%$) evaluated using 175B parameter GPT-based LLM.

Discussion (0). Sign in to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VLM-R$^3$: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought

    cs.CV 2025-05 conditional novelty 7.0 of 10

    VLM-R3 uses reinforcement learning to teach a multimodal model when and where to crop an image during its chain-of-thought, improving scores on several visual reasoning benchmarks.

  2. Learning How to Use Tools, Not Just When: Pattern-Aware Tool-Integrated Reasoning

    cs.AI 2025-09 reject novelty 6.0 of 10

    A two-stage pattern-aware tool-integrated reasoning method raises code usage and code-plus-correct metrics on math benchmarks, but the paper conflates Code@1 with problem-solving accuracy in its headline claims.

  3. SVAgent: AI Agent for Hardware Security Verification Assertion

    cs.CR 2025-07 conditional novelty 6.0 of 10

    SVAgent is a prompt-engineering framework that decomposes security requirements into sub-questions to generate SystemVerilog assertions with higher reported accuracy and consistency than direct LLM generation.

  4. Direct Behavior Optimization: Unlocking the Potential of Lightweight LLMs

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A gradient-free Monte Carlo tree search over JSON key-step plans produces few-shot demonstrations that let LLaMA3-8B and LLaMA3.2-3B outperform GPT-3.5 on most of seven BIG-Bench Hard tasks.

  5. Structured Pruning for Diverse Best-of-N Reasoning Optimization

    cs.CL 2025-06 reject novelty 6.0 of 10

    SPRINT learns to select which attention heads to prune per question, improving Pass@N over random head selection and multinomial sampling on MATH500 and GSM8K.

  6. More or Less Wrong: A Benchmark for Directional Bias in LLM Comparative Reasoning

    cs.CL 2025-06 conditional novelty 6.0 of 10

    Comparative words in prompts can shift LLM answers toward the framed direction in simple arithmetic comparisons, with demographic terms amplifying the effect.

  7. Towards Objective Fine-tuning: How LLMs' Prior Knowledge Causes Potential Poor Calibration?

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Fine-tuning on data aligned with an LLM's prior knowledge induces overconfidence, and CogCalib mitigates this by gating a calibration loss to known data.

  8. Explaining and Tuning Transformer-based LLMs in Arithmetic Tasks with Human Strategies

    cs.LG 2026-07 conditional novelty 5.0 of 10

    A small transformer learns addition, multiplication, and division by mastering simple digit subtasks first, and human teaching strategies lift its arithmetic accuracy to ~100%.

  9. A Replicate-and-Quantize Strategy for Plug-and-Play Load Balancing of Sparse Mixture-of-Experts LLMs

    cs.LG 2026-02 reject novelty 5.0 of 10

    Replicate-and-Quantize copies the busiest MoE expert as a quantized duplicate and compresses the least important expert, lowering a new Load-Imbalance Score by up to 1.4x while accuracy varies by roughly -1.2 to +3.0 points.

  10. Can Structured Templates Facilitate LLMs in Tackling Harder Tasks? : An Exploration of Scaling Laws by Difficulty

    cs.AI 2025-08 reject novelty 5.0 of 10

    Training on easy synthetic math data lowers accuracy on hard benchmarks, and the proposed SST framework, which teaches explicit procedural chains, aims to reverse that drop.

  11. Enhancing Chain-of-Thought Reasoning with Critical Representation Fine-tuning

    cs.CL 2025-07 conditional novelty 5.0 of 10

    CRFT selects critical internal representations via attention and saliency scores and fine-tunes only them, improving GSM8K accuracy over ReFT from 29.0% to 32.8% on LLaMA-2-7B.

  12. GSM-Plus-BN: A Perturbation-Based Benchmark for Bangla Mathematical Reasoning in Large Language Models

    cs.CL 2026-07 conditional novelty 4.0 of 10

    The paper releases GSM-Plus-BN, a human-verified Bengali translation of the GSM-Plus perturbed math benchmark, and reports accuracy baselines for six open LLMs under standard and CoT prompting.

  13. From Implicit Exploration to Structured Reasoning: Leveraging Guideline and Refinement for LLMs

    cs.AI 2025-09 conditional novelty 4.0 of 10

    A guideline-and-refinement prompting framework distills reasoning rules from past successes and failures, applies them stepwise with per-step correction, and reports gains over CoT, ReAct, ToT, Beats, and FoT on selec...

  14. We Need Knowledge Distillation for Solving Math Word Problems

    cs.CL 2025-07 reject novelty 4.0 of 10

    A 3-layer student transformer distilled from compressed BERT vectors retains roughly 90 percent of teacher performance on Math23K, according to the authors, though the teacher baseline is not shown.

  15. A Large Language Model-Empowered Agent for Reliable and Robust Structural Analysis

    cs.CL 2025-06 conditional novelty 4.0 of 10

    An LLM agent that reframes beam analysis as OpenSeesPy code generation reaches over 99 percent reliability on a small benchmark, but chiefly because the prompt contains a near-identical solved example.

  16. A Technical Survey of Reinforcement Learning Techniques for Large Language Models

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A survey of RL methods for LLMs that organizes the field by reward modeling, feedback source, and optimization strategy, with benchmark tables favoring a scalar-regression UNA variant over DPO and KTO in offline alignment.

  17. Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directions

    cs.CL 2025-06 conditional novelty 3.0 of 10

    LLM robustness research is organized into adversarial robustness, out-of-distribution robustness, and evaluation, with an accompanying GitHub collection of papers.

Pith tools