REVIEW 6 cited by
Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Scaling pre-training compute has proven effective for achieving mulitlinguality, but does the same hold for test-time scaling? In this work, we introduce MCLM, a multilingual math benchmark featuring competition-level problems in 55 languages. We test three test-time scaling methods-Outcome Reward Modeling (ORM), Process Reward Modeling (ORM), and Budget Forcing (BF)-on both Qwen2.5-1.5B Math and MR1-1.5B, a multilingual LLM we trained for extended reasoning. Our experiments show that using Qwen2.5-1.5B Math with ORM achieves a score of 35.8 on MCLM, while BF on MR1-1.5B attains 35.2. Although "thinking LLMs" have recently garnered significant attention, we find that their performance is comparable to traditional scaling methods like best-of-N once constrained to similar levels of inference FLOPs. Moreover, while BF yields a 20-point improvement on English AIME, it provides only a 1.94-point average gain across other languages-a pattern consistent across the other test-time scaling methods we studied-higlighting that test-time scaling may not generalize as effectively to multilingual tasks. To foster further research, we release MCLM, MR1-1.5B, and evaluation results.
Forward citations
Cited by 6 Pith papers
-
When to Vote, When to Rewrite: Disagreement-Guided Strategy Routing for Test-Time Scaling
A disagreement-guided routing framework dynamically selects among resolution, voting, and rewriting strategies for test-time scaling, delivering 3-7% accuracy gains with lower sampling cost on mathematical benchmarks.
-
When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMs
Hedged sampling, checklist-based one-pass selection (CHOPS), and cross-lingual MBR (X-MBR) improve multilingual LLM output quality when scaling from one to five samples.
-
MMATH: A Multilingual Benchmark for Mathematical Reasoning
A new multilingual math benchmark shows that reasoning models often respond in the wrong language, and English-reasoning training improves both accuracy and language consistency.
-
Controlling Language Confusion in Multilingual LLMs
ORPO fine-tuning, which explicitly penalizes disfavored language-mixed responses, nearly eliminates language confusion in Korean-generation LLMs without hurting QA accuracy.
-
Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
A taxonomy-driven survey arguing that reward design is the central mechanism shaping reliable LLM reasoning, with maps of reward paradigms, reward-hacking failure modes, and benchmark pitfalls.
-
AgentScope 1.0: A Developer-Centric Framework for Building Agentic Applications
AgentScope 1.0 packages the components needed to build, evaluate, and deploy LLM agent applications into one developer framework.
Discussion (0). Continue with ORCID to comment.