Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

Step-Opt: Boosting Optimization Modeling in LLMs through Iterative Data Synthesis and Structured Validation

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A fine-tuned 8B model trained on validated synthetic data beats GPT-4 on complex optimization-modeling benchmarks.

desk verdict Real contribution to LLM-for-OR data synthesis, but the headline SOTA margins rest on manually relabeled test sets that need clearer baseline alignment. read the letter →

arxiv 2506.17637 v1 pith:PC7MDKUM submitted 2025-06-21 cs.CL cs.LG

Yang Wu , Yifan Zhang , Yurong Wu , Yuran Wang , Junkai Zhang , Jian Cheng This is my paper · ORCID
classification cs.CLcs.LG
keywords optimizationmodelingoperationsresearchlargelanguagemodelssyntheticdatagenerationiterativeproblemstepwisevalidationfine-tuningmathematicalprogramming
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the bottleneck in LLM-based optimization modeling is not model size but the quality control of synthetic training data. It introduces Step-Opt-Instruct, which grows a small seed set of 260 operations-research problems into 4,464 training examples by iteratively making problems harder or shifting them to new domains, and by running each candidate through a four-stage validation pipeline before accepting it. Fine-tuning LLaMA-3-8B and Mistral-7B on this filtered data yields Step-Opt, reported as state-of-the-art on NL4OPT, MAMO, and IndustryOR, with a 17.01% micro-average improvement on the most difficult problems over the strongest baselines. A sympathetic reader would take the central claim to be that structured validation plus gradual problem refinement can make small open-source models competitive with, or better than, much larger proprietary models on complex OR tasks.

What carries the argument

The load-bearing mechanism is the Step-Opt-Instruct data-generation and filtering loop. Iterative Problem Generation uses five prompt-based evolvers: constraint modification, objective alteration, and parameter adjustment raise difficulty one small step at a time while preserving the seed problem's logical structure, while domain transformation and combination add breadth by moving a problem to a new application area or merging two problems. Stepwise Validation then runs four LLM-based checkers—description completeness, variable definition, constraint correctness (including dedicated Big-M and K-way selection checks), and program execution—and feeds identified errors back to the generator for revision, discarding any item that fails after the retry limit. The paper claims this loop is what keeps the synthetic dataset free of inherited errors, and that this data quality is what allows a 7B-to-8B model to surpass much larger prompt-driven systems on complex problems.

What would settle it

Re-run every baseline (ORLM, GPT-4 CoE, GPT-4 Reflexion, and the rest) on the exact corrected instances used for the reported numbers and compare against scores on the original uncorrected benchmarks; if the corrections move ORLM or GPT-4 accuracy by more than the claimed gaps, the state-of-the-art claim is an artifact of relabeling. Also train LLaMA-3-8B on an equal-sized unfiltered subset of the same generated data; if accuracy on MAMO ComplexLP stays near 61.61% without the stepwise validation filter, the filtering mechanism is not the cause of the gain.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that carefully filtered synthetic data, not prompt engineering, is what unlocks complex optimization modeling in open-source LLMs. Step-Opt-Instruct combines iterative problem generation—Complexity-Evolving (constraint modification, objective alteration, parameter adjustment) and Scope-Evolving (domain transformation, problem combination)—with a stepwise validation mechanism of four checkers covering description completeness, variable definitions, constraint implementation (with special prompts for Big-M and K-way selection), and executable program output. Only problems that pass every check enter the training set, preventing errors from propagating through later iterations. The resulting Step-Opt-LLaMA-3-8B reaches 84.49% on NL4OPT, 85.28% on MAMO EasyLP, 61.61% on MAMO ComplexLP, and 36.36% on IndustryOR, which the paper reports as state-of-the-art, with a 17.01% micro-average gain over baselines on complex problems. The paper also reports ablations showing that each evolving method contributes, that discarding the mathematical model from training data hurts even when token counts are equalized, and that Step-Opt-Instruct outperforms ORLM's OR-Instruct data when both are used to fine-tune the same backbone with the same number of examples.

Load-bearing premise

The reported results assume that the manual corrections to the benchmark labels—16 in NL4OPT, 78 in MAMO, and 50 corrections plus 23 removals in IndustryOR—are genuine fixes to ground truth rather than the authors' own modeling conventions, and that every baseline was scored under the same corrected set.

Editorial extensions

If this is right

  • Because generation and validation are automated, the pipeline removes the need for manual post-processing of training data, so scaling to more OR problem types should mainly cost API compute rather than expert labor.
  • If the benchmark results hold, organizations with sensitive operations data can fine-tune small local models instead of sending problem descriptions to proprietary APIs.
  • The ablation results imply that the composition of the synthetic data matters more than its raw size: dropping any one evolution method degrades performance, with domain transformation the most broadly damaging.
  • Keeping the mathematical model as an intermediate artifact in training data improves the generated solver code even when token counts are held fixed, indicating that the model is acting as a structured reasoning bridge.
  • The largest gains appear precisely on the hardest instances, so the method's value is concentrated where prompt-based methods currently fail.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that the same generate-check-retry-discard pattern could transfer to other structured-generation tasks such as SQL, formal proofs, or scheduling rules, where synthetic-data errors compound across iterations and the checker set would need to be re-specified per domain.
  • The reported manual benchmark corrections (16 in NL4OPT, 78 in MAMO, 50 corrections plus 23 removals in IndustryOR) change the evaluation set, so the state-of-the-art comparison is only meaningful if all baselines were re-run on the corrected instances; an independent re-scoring would be the decisive test.
  • Because 46.86% of generated samples were discarded by validation, the improvement could come from filtering, from the extra generation attempts, or both; training on an unfiltered equal-size sample would isolate the validation mechanism's causal contribution.
  • The authors' stated limitations—uncovered OR modeling techniques and untested other fields—suggest the checkers, not the generators, are the current ceiling; adding checkers for new technique families is the natural way to broaden applicability.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces Step-Opt-Instruct, a framework for synthesizing fine-tuning data for optimization modeling with LLMs. The framework combines iterative problem generation (complexity-evolving and scope-evolving prompts) with a stepwise validation mechanism (description, variable, constraint, and program checkers) to produce a dataset of 4,464 verified examples. The authors fine-tune LLaMA-3-8B and Mistral-7B with LoRA on this data, yielding Step-Opt models that reportedly achieve state-of-the-art accuracy on NL4OPT, MAMO EasyLP/ComplexLP, and IndustryOR. The paper also reports manual correction of benchmark labels (16 in NL4OPT, 78 in MAMO, 50 corrections plus 23 removals in IndustryOR) and provides ablations of the evolving-method components and of the inclusion of the mathematical-model component in training data.

Significance. If the reported results are valid, this is a useful contribution to automated OR modeling: it demonstrates a fully automated data synthesis pipeline that avoids manual post-processing, and it improves open-source 7B/8B models to the point of outperforming GPT-4-based prompting on several benchmarks. The strengths include a clear framework description, release of code and data, and multiple ablations (evolving methods, model vs. program only, comparison with ORLM). The central risk is evaluation integrity: the manual relabeling of test sets by the authors, without independent annotation, and the reliance on LLM-based validation for generated data make the headline SOTA gains difficult to interpret as objective model-capability improvements. The paper also omits an ablation of the validation mechanism in the data-generation pipeline, which is central to its claimed contribution.

major comments (3)
  1. [§4.1, Table 1] The reported state-of-the-art results depend critically on the manual correction of test-set labels (16 in NL4OPT, 78 in MAMO, 50 corrections plus 23 removals in IndustryOR). The paper never explicitly states that every baseline in Table 1 was re-evaluated on the corrected instances; only tag-BART is marked as copied from the original paper. If the prompt-engineering baselines and ORLM were scored on the original labels while Step-Opt was scored on the corrected ones, the reported gains—for example, the 21.33% lead on MAMO ComplexLP and the derived 17.01% micro-average improvement on complex problems—would be inflated by an evaluation-set change rather than by model capability. The authors should release the corrected instances and state clearly that all methods were scored on the same corrected test set, preferably with independent annotation or a detailed per-instance changelog.
  2. [§3.3] The stepwise validation mechanism uses LLM-based checkers to assess whether the generated problem description, variables, constraints, and program are mutually consistent and consistent with the problem text. This verifies internal consistency, not correctness against an external solver or ground truth; the program checker only executes the code to detect runtime errors, not modeling errors. Because the same model family (GPT-4) both generates and validates the data, the claim that the pipeline produces 'rigorously verified' data is overstated. This circularity weakens the inference that the observed performance gains come from higher data quality rather than from distributional alignment between the generation/validation model and the downstream fine-tuning objective. The authors should temper this claim and, if possible, incorporate a solver-based verification step (e.g., comparing objective values or feasibility certificates) for at least a sample of generated instances.
  3. [§4.3, Table 2] The ablation study on evolving methods trains on a randomly sampled 2,000-example subset, whereas the main Step-Opt model uses 4,464 examples. Therefore the ablation numbers in Table 2 are not directly comparable to Table 1, and the measured drop when removing a method could be partially a data-subsampling effect. In addition, the paper never ablates the stepwise validation mechanism itself from the data-generation pipeline: there is no comparison of models trained on data generated with versus without the checkers. Since the validation mechanism is advertised as a core contribution, its absence from the ablations leaves a gap in the evidence for its claimed role in preventing error propagation.
minor comments (6)
  1. [Abstract] The phrase 'complex problem' should be 'complex problems'.
  2. [§4.1] The sentence 'Answers were manually revised when needed' is vague; the subsequent paragraph gives counts, but it would help to state explicitly that all baselines were re-run on the corrected versions (or, if not, to list which were not).
  3. [Appendix A.4] There is a duplicated phrase 'the following: the following:' in the domain-transformation prompt.
  4. [Appendix A.5] The odd accent in 'Érror´' appears to be a typographical artifact; it should be 'Error'.
  5. [§4.3] The phrase 'we randomly sample 2,000 examples without specific methods' is ambiguous; it could mean the sample is not stratified by evolving method, or that no evolving method is applied. Please clarify.
  6. [Eq. (1)] The denominator is written as 'g+ϵ'; if g can be zero or negative, the comparison rule should use |g|+ϵ and should be stated explicitly.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation chain; manual benchmark relabeling is a comparability concern, not a by-construction reduction.

full rationale

The paper's claimed derivation chain is external and self-contained rather than self-referential. Step-Opt-Instruct generates problems with GPT-4, validates them through separate description, variable, constraint, and program checkers, and fine-tunes LLaMA-3-8B and Mistral-7B on the surviving 4,464 examples. The central claims are then tested on external benchmarks (NL4OPT, MAMO, IndustryOR). The manual corrections in Sec. 4.1 (16, 78, and 50 corrections plus 23 removals) are a genuine evaluation-set validity concern: unless every baseline was re-scored on the same corrected labels, the reported SOTA margins, especially the 21.33% lead on MAMO ComplexLP, could be inflated. However, this is a transparency and comparability issue, not circularity: the paper does not define the corrected ground truth from Step-Opt's own outputs, and there is no equation or construction forcing the reported numbers from the relabeling. Independent grounding exists in the ablations (Tables 2-4) and the controlled 3K-example comparison with ORLM (Table 3), which show gains from the framework's components beyond any relabeling effect. No self-citation chain, uniqueness import, or ansatz-smuggling pattern is present. Accordingly, the paper is not circular in the sense targeted by this analysis, though the benchmark-correction procedure should be reported with full transparency.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The framework does not fit numerical parameters to data; the central empirical result depends instead on domain assumptions about the reliability of LLM checkers, the validity of the seed data, and the correctness of the authors' manual benchmark relabeling. Standard training hyperparameters (learning rate, epochs, LoRA rank) are reported in Appendix A.8 but are not treated as scientific free parameters.

assumptions (4)
  • domain assumption LLM-based checkers can reliably detect errors in OR models and programs.
    Sec. 3.3 relies on GPT-4-based Description, Variables, Constraints, and Program Checkers; there is no solver-verified ground truth for the 4,464 generated instances, so the data quality claim rests on these checkers being correct.
  • domain assumption The 260 seed cases are correct and representative.
    Sec. 3.1 defines the seed dataset; any errors or narrow coverage in the seeds propagate through all generations.
  • ad hoc to paper Manual benchmark corrections are true errors.
    Sec. 4.1 and Appendix A.2 list corrections and removals across NL4OPT, MAMO, and IndustryOR without an independent annotation study; the corrected labels determine all reported accuracy numbers.
  • ad hoc to paper Evolving prompts preserve problem validity and real-world plausibility.
    Appendix A.3 instructs GPT-4 to modify constraints, objectives, parameters, or domains while retaining logical structure, but no formal feasibility guarantee is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Step-Opt: Boosting Optimization Modeling in LLMs through Iterative Data Synthesis and Structured Validation." pith.science (2026). https://pith.science/paper/PC7MDKUM

@misc{pith2026250617637,
  author       = {Pith},
  title        = {Pith review of: Step-Opt: Boosting Optimization Modeling in LLMs through Iterative Data Synthesis and Structured Validation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PC7MDKUM}},
  note         = {Machine review of arXiv:2506.17637}
}
read the original abstract

Large Language Models (LLMs) have revolutionized various domains but encounter substantial challenges in tackling optimization modeling tasks for Operations Research (OR), particularly when dealing with complex problem. In this work, we propose Step-Opt-Instruct, a framework that augments existing datasets and generates high-quality fine-tuning data tailored to optimization modeling. Step-Opt-Instruct employs iterative problem generation to systematically increase problem complexity and stepwise validation to rigorously verify data, preventing error propagation and ensuring the quality of the generated dataset. Leveraging this framework, we fine-tune open-source LLMs, including LLaMA-3-8B and Mistral-7B, to develop Step-Opt--a model that achieves state-of-the-art performance on benchmarks such as NL4OPT, MAMO, and IndustryOR. Extensive experiments demonstrate the superior performance of Step-Opt, especially in addressing complex OR tasks, with a notable 17.01\% improvement in micro average accuracy on difficult problems. These findings highlight the effectiveness of combining structured validation with gradual problem refinement to advance the automation of decision-making processes using LLMs.The code and dataset are available at https://github.com/samwu-learn/Step.

Figures

Figures reproduced from arXiv: 2506.17637 by the authors.

Figure 1
Figure 1. Examples of Iterative Problem Generation. It includes two types of methods: Complexity-Evolving, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The framework of Step-Opt-Instruct. Each iteration begins by sampling seed data from an initial dataset. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Performance comparison of various methods on easy and complex datasets. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Examples of training data [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PEARL: Solver-in-the-Loop Interactive Optimization Modeling from Natural Language

    cs.AI 2026-05 reject novelty 7.0 of 10

    Training an LLM as a multi-turn agent that runs and repairs solver code raises verified optimization solve rates, with the 4B PEARL model outperforming DeepSeek-V3.2-685B in aggregate.

  2. A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving

    cs.NE 2025-09 conditional novelty 4.0 of 10

    A literature survey that classifies LLM-based optimization research into modeling and solving, with solving divided into LLMs as optimizers, low-level components, and high-level managers.

Reference graph

Works this paper leans on

34 extracted references · 8 canonical work pages · cited by 2 Pith papers

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Ali AhmadiTeshnizi, Wenzhi Gao, and Madeleine Udell. 2024. Optimus: Scalable optimization modeling with (mi) lp solvers and large language models. arXiv preprint arXiv:2402.10172

    arXiv 2024

  3. [3]

    AI@Meta. 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card

    2024

  4. [4]

    Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, and Weizhu Chen. 2023. Learning from mistakes makes llm better reasoner. arXiv preprint arXiv:2310.20689

    arXiv 2023

  5. [5]

    Ali Belgacem, Kadda Beghdad-Bey, Hassina Nacer, and Sofiane Bouznad. 2020. Efficient dynamic resource allocation method for cloud computing environment. Cluster Computing, 23(4):2871--2889

    work page 2020

  6. [6]

    Dimitris Bertsimas, Jack Dunn, and Nishanth Mundru. 2019. Optimal prescriptive trees. INFORMS Journal on Optimization, 1(2):164--183

    work page 2019

  7. [7]

    Saumya Gandhi, Ritu Gala, Vijay Viswanathan, Tongshuang Wu, and Graham Neubig. 2024. Better synthetic data by retrieving and transforming existing datasets. arXiv preprint arXiv:2404.14361

    arXiv 2024

  8. [8]

    Dongdong Ge, Qi Huangfu, Zizhuo Wang, Jian Wu, and Yinyu Ye. 2022. Cardinal optimizer (copt) user guide. arXiv preprint arXiv:2208.14314

    arXiv 2022

Show all 34 references
  1. [9]

    Gurobi Optimization, LLC . 2024. https://www.gurobi.com Gurobi Optimizer Reference Manual

    2024

  2. [10]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  3. [11]

    Xuhan Huang, Qingning Shen, Yan Hu, Anningzhe Gao, and Benyou Wang. 2024. Mamo: a mathematical modeling benchmark with solvers. arXiv preprint arXiv:2405.13144

    2024 arXiv

  4. [12]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, and 1 others. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  5. [13]

    Nickvash Kani and Neeraj Gangwar. 2022. Tagged input and decode all-at-once strategy. https://github.com/MLPgroup/nl4opt-generation

    2022

  6. [14]

    Beibin Li, Konstantina Mellou, Bo Zhang, Jeevan Pathuri, and Ishai Menache. 2023 a . Large language models for supply chain optimization. arXiv preprint arXiv:2307.03875

    2023 arXiv

  7. [15]

    Qingyang Li, Lele Zhang, and Vicky Mak-Hau. 2023 b . Synthesizing mixed-integer linear programming models from natural language descriptions. arXiv preprint arXiv:2311.15271

    2023 arXiv

  8. [16]

    Seokjin Oh, Su Ah Lee, and Woohwan Jung. 2023. Data augmentation for neural machine translation using generative language model. arXiv preprint arXiv:2307.16833

    2023 arXiv

  9. [17]

    Yan Pan, Davide Cadamuro, and Georg Groh. 2023. Data-augmented task-oriented dialogue response generation with domain adaptation. In Proceedings of the 37th Pacific Asia Conference on Language, Information and Computation, pages 96--106

    2023

  10. [18]

    Jo \ a o Luiz Junho Pereira, Guilherme Ant \^o nio Oliver, Matheus Brendon Francisco, Sebastiao Simoes Cunha Jr, and Guilherme Ferreira Gomes. 2022. A review of multi-objective optimization: methods and algorithms in mechanical engineering problems. Archives of Computational M...

    2022

  11. [19]

    Ganesh Prasath and Shirish Karande. 2023. Synthesis of mathematical programs from natural language specifications. arXiv preprint arXiv:2304.03287

    2023 arXiv

  12. [20]

    Rindranirina Ramamonjison, Timothy Yu, Raymond Li, Haley Li, Giuseppe Carenini, Bissan Ghaddar, Shiqi He, Mahdi Mostajabdaveh, Amin Banitalebi-Dehkordi, Zirui Zhou, and 1 others. 2023. Nl4opt competition: Formulating optimization problems based on their natural language descri...

    2023

  13. [21]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2024. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36

    2024

  14. [22]

    Zhengyang Tang, Chenyu Huang, Xin Zheng, Shixi Hu, Zizhuo Wang, Dongdong Ge, and Benyou Wang. 2024. Orlm: Training large language models for optimization modeling. arXiv preprint arXiv:2405.17743

    2024 arXiv

  15. [23]

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Stanford alpaca: An instruction-following llama model

    2023

  16. [24]

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560

    2022 arXiv

  17. [25]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  18. [26]

    Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. 2024. Magicoder: Empowering code generation with oss-instruct. In Forty-first International Conference on Machine Learning

    2024

  19. [27]

    Ziyang Xiao, Dongxiang Zhang, Yangjun Wu, Lilin Xu, Yuan Jessica Wang, Xiongwei Han, Xiaojin Fu, Tao Zhong, Jia Zeng, Mingli Song, and 1 others. 2023. Chain-of-experts: When llms meet complex operations research problems. In The Twelfth International Conference on Learning Rep...

    2023

  20. [28]

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. 2024. Wizardlm: Empowering large pre-trained language models to follow complex instructions. In The Twelfth International Conference on Learning Representations

    2024

  21. [29]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024 a . Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

  22. [30]

    Zhicheng Yang, Yinya Huang, Wei Shi, Liang Feng, Linqi Song, Yiwei Wang, Xiaodan Liang, and Jing Tang. 2024 b . Benchmarking llms for optimization modeling and enhancing reasoning via reverse socratic synthesis. arXiv preprint arXiv:2407.09887

    2024 arXiv

  23. [31]

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. http://arxiv.org/abs/2403.13372 Llamafactory: Unified efficient fine-tuning of 100+ language models . In Proceedings of the 62nd Annual Meeting of the Association for Compu...

    2024 arXiv

  24. [32]

    Kun Zhou, Beichen Zhang, Jiapeng Wang, Zhipeng Chen, Wayne Xin Zhao, Jing Sha, Zhichao Sheng, Shijin Wang, and Ji-Rong Wen. 2024. Jiuzhang3. 0: Efficiently improving mathematical reasoning by training small data synthesis models. arXiv preprint arXiv:2405.14365

    2024 arXiv

  25. [33]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  26. [34]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.