Pith. sign in

REVIEW 6 cited by

How well do Large Language Models perform in Arithmetic tasks?

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2304.02015 v1 pith:J2ZGI2I7 submitted 2023-03-16 cs.CL cs.AI

Zheng Yuan , Hongyi Yuan , Chuanqi Tan , Wei Wang , Songfang Huang This is my paper · ORCID
classification cs.CLcs.AI
keywords arithmeticlanguagelargemodelsmathproblemsstepabilities
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models have emerged abilities including chain-of-thought to answer math word problems step by step. Solving math word problems not only requires abilities to disassemble problems via chain-of-thought but also needs to calculate arithmetic expressions correctly for each step. To the best of our knowledge, there is no work to focus on evaluating the arithmetic ability of large language models. In this work, we propose an arithmetic dataset MATH 401 to test the latest large language models including GPT-4, ChatGPT, InstrctGPT, Galactica, and LLaMA with various arithmetic expressions and provide a detailed analysis of the ability of large language models. MATH 401 and evaluation codes are released at \url{https://github.com/GanjinZero/math401-llm}.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics

    cs.RO 2026-02 conditional novelty 6.0 of 10

    The log-probability a VLM assigns to 'True' for 'does this video prefix complete the task?' is used as a zero-shot dense progress reward that outperforms GVL on open-source models.

  2. Ignore the KL Penalty! Boosting Exploration on Critical Tokens to Enhance RL Fine-Tuning

    cs.CL 2025-02 conditional novelty 6.0 of 10

    A certainty-weighted KL penalty that down-weights the penalty on low-confidence tokens improves RL fine-tuning exploration for arithmetic in GPT-2.

  3. Towards Interpretable Time Series Foundation Models

    cs.CL 2025-07 conditional novelty 5.0 of 10

    After fine-tuning on 180 synthetic mean-reverting series annotated by a large multimodal model, small Qwen models can describe trend direction, noise intensity, and extremum location in natural language.

  4. Evaluating the Meta- and Object-Level Reasoning of Large Language Models for Question Answering

    cs.CL 2025-02 conditional novelty 5.0 of 10

    LLMs frequently generate rational plans for multi-step questions but often fail to produce answers, and a new Franklin dataset is especially hard for them.

  5. Code Simulation as a Proxy for High-order Tasks in Large Language Models

    cs.LG 2025-02 conditional novelty 5.0 of 10

    LLM performance on naturalistic reasoning tasks tracks performance on equivalent Python code simulation, but the effect is partly driven by pattern matching and memorization rather than faithful execution.

  6. Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges

    cs.CL 2026-05 unverdicted novelty 4.0 of 10

    A structured survey of LLM mathematical reasoning that unifies dataset taxonomies, reviews architectures and training strategies, and highlights the gap between answer accuracy and process-level verification.

Pith tools