REVIEW 7 cited by
LLMs for Mathematical Modeling: Towards Bridging the Gap between Natural and Mathematical Languages
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large Language Models (LLMs) have demonstrated strong performance across various natural language processing tasks, yet their proficiency in mathematical reasoning remains a key challenge. Addressing the gap between natural and mathematical language requires advanced reasoning capabilities, approaching those of Artificial General Intelligence (AGI). However, the evaluation remains challenging, as perfectly representing reality is inherently elusive, and traditional methods like manual or direct comparison of mathematical statements (Ramamonjison et al., 2023) are insufficient for assessing true modeling ability. We propose a process-oriented framework to evaluate LLMs' ability to construct mathematical models, using solvers to compare outputs with ground truth. Introducing Mamo, a benchmark with 1,209 questions covering ordinary differential equations, linear programming, and mixed-integer linear programming, we enable automatic evaluation of modeling accuracy. The results show that existing LLMs struggle with complex mathematical modeling tasks, with larger models demonstrating superior performance, while open-source models remain competitive in simpler cases but still fall short of proprietary models in more challenging problems.
Forward citations
Cited by 7 Pith papers
-
Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
A sound, threshold-free battery of optimization-theoretic tests can catch unfaithful LLM-generated MILP models while never flagging faithful ones, and provably cannot catch certain error classes.
-
PEARL: Solver-in-the-Loop Interactive Optimization Modeling from Natural Language
Training an LLM as a multi-turn agent that runs and repairs solver code raises verified optimization solve rates, with the 4B PEARL model outperforming DeepSeek-V3.2-685B in aggregate.
-
SAC-Opt: Semantic Anchors for Iterative Correction in Optimization Modeling
A semantic-anchor alignment loop that reconstructs and verifies constraints from generated solver code lifts average modeling accuracy by 7.8 percentage points over seven datasets.
-
DualSchool: How Reliable are LLMs for Optimization Education?
DualSchool shows that open LLMs explain dualization well but achieve at most 47.8% accuracy on generating correct duals, and fail at verification and error classification.
-
SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD
An Ascend-NPU training stack reaches 34.22% MFU on DeepSeek-V4-Pro, and a solver-verified CPT+SFT recipe raises OR benchmark averages to 71.81% (Flash) and 77.33% (Pro).
-
Uncertainty-Aware Simulation-Based Inference for Operations Research with Large Language Models
Lookahead resampling with entropy- and power-based rewards steers LLM decoding toward OR formulations whose short simulated continuations are most concentrated, giving reported pass@1 gains not yet separated from adde...
-
A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving
A literature survey that classifies LLM-based optimization research into modeling and solving, with solving divided into LLMs as optimizers, low-level components, and high-level managers.
Discussion (0). Continue with ORCID to comment.