Pith. sign in

REVIEW 4 major objections 6 minor 57 references

Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Optimization problem-solving can run end-to-end through evolutionary AI agents.

desk verdict A coherent position paper with a clean product-space framing and two honest-but-partial case studies; the headline claim about navigating P and F outstrips the evidence, which only covers A and H. read the letter →

arxiv 2505.04354 v1 pith:E4BI6SQS submitted 2025-05-07 math.OC cs.AI

Wenhao Li , Bo Jin , Mingyi Hong , Changhong Lu , Xiangfeng Wang This is my paper · ORCID
classification math.OCcs.AI
keywords evolutionaryagenticworkflowfoundationmodelsoptimizationspaceautomaticalgorithmdesignhyperparameteradaptationADMMcloudresourceschedulingAutoML
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This position paper argues that the main bottleneck in optimization practice is human expertise, and that an evolutionary agentic workflow -- foundation models proposing candidate problems, formulations, algorithms, and hyperparameters, with evolutionary search selecting and refining them -- can remove that bottleneck. It formalizes the search target as the optimization space $O := \mathcal{P} \otimes \mathcal{F} \otimes \mathcal{A} \otimes \mathcal{H}$, the product of problem, formulation, algorithm, and hyperparameter spaces, so that solving a problem becomes searching for a good point in that space. Two case studies support the position: an evolved virtual-machine scheduling heuristic beats a best-fit baseline on training and test scenarios, and an evolved adaptive ADMM step-size rule reaches the same accuracy in far fewer iterations than an expert-designed scheme, including when transferred to new problems. A sympathetic reader would care because, if the position is right, optimization would no longer depend on a small pool of specialists and academic advances could reach industry without a long expert translation step. The paper itself notes that the workflow still lacks built-in theoretical verification and currently has a high inference cost.

What carries the argument

The central object is the optimization space $O := \mathcal{P} \otimes \mathcal{F} \otimes \mathcal{A} \otimes \mathcal{H}$, where each subspace may be a singleton or an infinite-dimensional functional space, and 'solving an optimization problem' is rewritten as 'searching for a point in $O$'. The argument is carried by a closed feedback loop: foundation agents (built from memory, reasoning, world-modeling, and action modules) generate candidate points; the reasoning module, trained on chain-of-thought data through supervised fine-tuning and rule-based reinforcement learning, filters and validates them; and evolutionary search maintains distributed island populations, clusters solutions to preserve diversity, and uses knowledge from memory to guide the next generation. This machinery is what converts creative, possibly hallucinated proposals into a disciplined search that improves over generations.

What would settle it

Run the identical evolutionary loop with the foundation-model proposer replaced by random mutation of existing candidate code and parameters, keeping the evaluation budget and selection pressure the same; if random search matches the agentic version on the VM-scheduling and ADMM benchmarks, the claim that model knowledge drives the improvement would be refuted. A second check: if more than roughly 90 percent of the model's proposals are rejected as infeasible by the reasoning module across a broad sample of problems, the evolutionary bootstrap cannot work regardless of selection.

Watch

Extended reading notes

Core claim

The central claim is that an evolutionary agentic workflow can autonomously navigate the entire optimization space $O := \mathcal{P} \otimes \mathcal{F} \otimes \mathcal{A} \otimes \mathcal{H}$, not merely tune hyperparameters or select algorithms inside a fixed model. Foundation agents provide broad-domain knowledge, context-sensitive parsing, and creative generation; a reasoning module filters candidates for mathematical and operational validity; and evolutionary algorithms manage population diversity, memory of successful patterns, and selection pressure. The case studies hold the formulation and hyperparameter spaces to singletons in order to isolate single subspaces: the cloud-scheduling study evolves the algorithm space, while the ADMM study evolves the hyperparameter space. In both, the evolved artifact outperforms the expert baseline and generalizes to unseen instances. The authors intend these results as evidence for the general mechanism, which they argue can search over problem descriptions and formulations as well.

Load-bearing premise

The load-bearing premise is that foundation models, after filtering by the reasoning module, generate candidates -- formulations, algorithms, and hyperparameters -- that are good enough, often enough, for evolutionary selection to improve upon, despite documented failures of such models at multi-step mathematical reasoning.

Editorial extensions

If this is right

  • A natural-language optimization problem could be carried from formulation to tuned solver with minimal human intervention, lowering the specialist barrier the paper identifies.
  • The workflow goes beyond AutoML: because the formulation space is part of $O$, the system can reformulate objectives and constraints, not merely pick among fixed algorithm and hyperparameter configurations.
  • Evolved artifacts can transfer: the VM scheduling algorithm beats the baseline on sizes it was not trained on, and the ADMM step-size rules trained on two problems accelerate other problems in the same library.
  • Memory, retrieval, fine-tuning, and continual learning could let the workflow remain effective as industrial problem specifications shift over time.
  • If the position is correct, the relevant comparison class broadens: AutoML, learning-to-optimize, meta-optimization, and algorithm configuration become components or special cases of a single search over $O$.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper leaves implicit is evolving the formulation space itself: starting from one natural-language description, the loop should move among LP, network-flow, assignment, and set-cover models rather than holding the formulation fixed as a singleton.
  • If the position is right, the bottleneck shifts from scarce experts to compute and verification: the workflow's industrial viability will be governed by inference cost and by automated proof or validity checking of generated proposals, both of which the paper lists as open problems.
  • The evolutionary framing suggests the approach may tolerate frequent hallucinations: as long as the reasoning filter passes a small but steady share of viable candidates, selection pressure could compensate, so performance should degrade gradually rather than collapse as proposal quality worsens -- a direct empirical prediction.
  • The authors do not address the no-free-lunch direction: the claim is most plausible for structured families of industrial problems where foundation models encode transferable knowledge, and least plausible for entirely novel problem structures where random search is the benchmark.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper is a position paper arguing that optimization problem solving can be automated through "evolutionary agentic workflows" that combine foundation models (FMs) with evolutionary search. The authors define an optimization space O as the product of problem space P, formulation space F, algorithm space A, and hyperparameter space H, and propose that an agentic workflow can navigate all four subspaces. Two case studies are presented: cloud VM scheduling, where the workflow evolves a heuristic that outperforms BestFit, and ADMM step-size tuning, where evolved adaptive update rules reduce iteration counts on LibADMM benchmarks. The paper also includes a design for the agent architecture, a review of related work, and a discussion of limitations and alternative views.

Significance. If the central claim were established, the framework would be important: it would replace human expert effort in optimization modeling and algorithm design with automated, adaptive search, potentially accelerating deployment of advanced optimization methods in industry. The paper makes a useful conceptual contribution by unifying problem formulation, algorithm design, and hyperparameter tuning in a single space O and by proposing an explicit agent architecture (memory, reasoning, world modeling, action). It also provides two concrete case studies with transfer tests, which go beyond pure speculation, and it candidly discusses the current lack of theoretical verification and high inference cost. However, the empirical evidence is preliminary and narrow: both case studies fix P and F to singletons, and the reported improvements lack statistical rigor and reproducibility details. The strength of the position therefore rests more on the plausibility of the component-wise literature cited in §3.6 than on the presented demonstrations.

major comments (4)
  1. [§2 (Eq. 1) and §4.1/§4.2] The central claim, stated in the abstract and conclusion, is that the workflow autonomously navigates the full optimization space O = P⊗F⊗A⊗H. However, both case studies explicitly reduce the problem and formulation spaces to singletons: §4.1 says "we fix the problem formulation and hyperparameter settings to isolate the algorithm design space," and §4.2 says "we consider a fixed problem formulation and algorithm design... spaces for problem formulation and algorithm design are singletons." Thus the presented evidence supports navigation only over the algorithm space A (VM scheduling) and over adaptive rules in H (ADMM). This is load-bearing because if P and F cannot be navigated, the paradigm reduces to AutoML/AAC over A and H, and the end-to-end claim fails. The paper should either provide evidence for P and F navigation or explicitly reframe the claim as a conjecture for those subspaces, supported by the separate component-wise literature in §3.6.
  2. [§4.2, Tables 2 and 3] The ADMM case study reports single-point iteration counts without standard deviations, statistical tests, or details of multiple runs. Since the performance comparison is a central piece of evidence, the lack of repeated trials leaves open the possibility that the results are due to overfitting or to idiosyncrasies of the specific problems. Additionally, the paper does not state what convergence properties, if any, are preserved by the evolved update rules; §5 acknowledges that "foundation agents lack built-in mechanisms for theoretical verification," but the presented demonstration would be considerably stronger if it included at least an empirical convergence check and a comparison of run-to-run variability.
  3. [§3.1.2 and §4.1] The reasoning module described in §3.1.2—CoT data construction, domain-focused SFT, and rule-based RL—is presented as a component of the framework, but it is not clear whether this pipeline was actually implemented in the case studies. §4.1 mentions "300 training epochs" and the use of DeepSeek-Coder, but does not specify which of the proposed training steps were performed. This conflation of proposed design and implemented system makes it difficult to assess what the experiments actually demonstrate. The authors should clearly state which components of the architecture (memory, reasoning, world modeling, action) were used in each case study and how the reasoning module was instantiated.
  4. [§4.1] The description of the VM scheduling experiment omits critical details of the evolutionary search: population size, number of generations, mutation/crossover operators, LLM sampling temperature, selection mechanism, and the exact fitness evaluation procedure. Without these, the reader cannot assess whether the reported improvement over BestFit is robust or a result of particular search settings. The paper should report these settings, along with multiple random seeds and standard deviations, to make the empirical claims reproducible.
minor comments (6)
  1. [Figure 4 caption] The caption contains typographical errors: "perforcement" should be "performance," and "traing" should be "training."
  2. [Table 2 caption] The caption says "Iterative number" and "tolenence"; these should be "Number of iterations" and "tolerance."
  3. [§2] The word "summerized" should be "summarized."
  4. [§3.1.4] The phrase "atlasing the current solution approach" is unclear; presumably "adjusting" or "updating" was intended.
  5. [§5] The section opens with "there remain two key challenges" but then lists three (theoretical verification, inference cost, and scalability); please renumber or rephrase.
  6. [§4.2, Eq. (7)] The definitions of primal and dual residuals appear swapped relative to the standard ADMM convention: p_k is defined as ρ AᵀB(z^k−z^{k−1}) (usually the dual residual) and d_k as Ax^k+Bz^k−c (usually the primal residual). Please clarify the notation to avoid confusion with the literature cited.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claim is a position supported by externally benchmarked case studies, and Eq. (1) is a definitional decomposition rather than a self-contained derivation.

full rationale

The paper makes no mathematical derivation whose conclusion is secretly an input. Eq. (1), O := P⊗F⊗A⊗H, is an explicit definition of an optimization space, not a derived result that presupposes the conclusion. The two case studies deliberately restrict the searched subspaces (§4.1 fixes the problem formulation and hyperparameter settings; §4.2 fixes the problem formulation and algorithm design), and their reported evidence compares generated algorithms and hyperparameter rules against external baselines (BestFit and ADMM-expert) and includes transfer tests to held-out scenarios and LibADMM problems. No fitted constant is renamed as a prediction, and no uniqueness theorem or prior-work ansatz is invoked to force the proposed workflow. The few references to co-authored prior work (e.g., Sheng et al. 2022 for ML scheduling, Hong et al. 2015 for ADMM-related formulations) are background citations, not load-bearing support for the paper's central claim. The paper also explicitly concedes in Section 5 that foundation agents lack built-in theoretical verification and in Section 6 that LLM reasoning limitations and evolutionary-method criticisms are substantive alternatives. Extrapolation from problem-space-singleton case studies to the full four-subspace navigation claim is a scope limitation or correctness risk, not circularity. Therefore no circular step is present.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The paper defines an optimization space O := P x F x A x H and proposes that foundation agents plus evolutionary search can navigate it. The central position rests on domain assumptions about the reliability of LLM-generated candidates and on the unproven convergence of evolved hyperparameter rules, together with undisclosed experimental hyperparameters. No new physical or formal entities are introduced beyond the workflow concept itself.

free parameters (1)
  • Evolutionary workflow hyperparameters (population size, mutation operators, LLM temperature, selection pressure… = not reported
    The two case studies depend on these hand-chosen settings, but no values are given in Sections 4.1 or 4.2, so readers cannot assess sensitivity or reproduce the runs.
assumptions (3)
  • domain assumption The optimization space factorizes as O := P x F x A x H and is searchable across independent subspaces.
    Eq. (1) defines this product and the paper assumes navigating it solves real optimization problems, but both case studies fix all but one subspace, so full-space searchability is assumed rather than shown.
  • domain assumption Foundation-agent-generated candidates are sufficiently correct that the reasoning module and evolutionary selection can prune errors and improve them.
    Section 3.1.2 relies on a dedicated reasoning module to filter hallucinations, while Section 6 itself cites evidence that LLMs fail at multi-step reasoning. The workflow cannot bootstrap if most proposals are invalid.
  • ad hoc to paper Evolved ADMM beta-update rules preserve convergence for the problems tested.
    Section 4.2 reports fewer iterations but provides no convergence proof for NewADMM, and Section 5 explicitly admits foundation agents lack built-in mechanisms for theoretical verification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows." pith.science (2026). https://pith.science/paper/E4BI6SQS

@misc{pith2026250504354,
  author       = {Pith},
  title        = {Pith review of: Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E4BI6SQS}},
  note         = {Machine review of arXiv:2505.04354}
}
read the original abstract

This position paper argues that optimization problem solving can transition from expert-dependent to evolutionary agentic workflows. Traditional optimization practices rely on human specialists for problem formulation, algorithm selection, and hyperparameter tuning, creating bottlenecks that impede industrial adoption of cutting-edge methods. We contend that an evolutionary agentic workflow, powered by foundation models and evolutionary search, can autonomously navigate the optimization space, comprising problem, formulation, algorithm, and hyperparameter spaces. Through case studies in cloud resource scheduling and ADMM parameter adaptation, we demonstrate how this approach can bridge the gap between academic innovation and industrial implementation. Our position challenges the status quo of human-centric optimization workflows and advocates for a more scalable, adaptive approach to solving real-world optimization problems.

Figures

Figures reproduced from arXiv: 2505.04354 by the authors.

Figure 1
Figure 1. Comparative frameworks in optimization problem solving: human expertise and [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Automate optimization formulation, algorithmic design and hyperparameter [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Evolutionary agentic workflow for VM scheduling. [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The perforcement of the discovered algorithm during traing on scenario with VM [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Evolutionary agentic workflow for ADMM. where L(x, z, λ) = f(x) + g(z) − λ T (Ax + Bz − c) + β 2 ∥Ax + Bz − c∥ 2 2 , (6) where λ denotes the dual variable (or Lagrange multiplier) concerning the linear constraint, and β denotes the penalty parameter. In general, the pa…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 5 canonical work pages

  1. [9]

    SELA: Tree-search enhanced llm agents for automated machine learning.arXiv:2410.17238,

    Chi, Y., Lin, Y., Hong, S., Pan, D., Fei, Y., Mei, G., Liu, B., Pang, T., Kwok, J., Zhang, C., et al. SELA: Tree-search enhanced llm agents for automated machine learning.arXiv:2410.17238,

    arXiv

  2. [10]

    A survey of chain of thought reasoning: Advances, frontiers and future

    Chu, Z., Chen, J., Chen, Q., Yu, W., He, T., Wang, H., Peng, W., Liu, M., Qin, B., and Liu, T. A survey of chain of thought reasoning: Advances, frontiers and future. arXiv:2309.15402,

    arXiv

  3. [12]

    Artificial intelligence for operations research: Revolutionizing the operations research process

    Fan, Z., Ghaddar, B., Wang, X., Xing, L., Zhang, Y., and Zhou, Z. Artificial intelligence for operations research: Revolutionizing the operations research process. arXiv:2401.03244,

    arXiv

  4. [13]

    MIPLIB 2017: Data-driven compilation of the 6th mixed-integer programming library

    20 Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows Gleixner, A., Hendel, G., Gamrath, G., Achterberg, T., Bastubbe, M., Berthold, T., Christophel, P ., Jarck, K., Koch, T., Linderoth, J., et al. MIPLIB 2017: Data-driven compilation of the 6th mixed-integer programming library. Mathematical Programming Computation, 13(3): 443–490,

    work page 2017

  5. [15]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P ., Bi, X., et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv:2501.12948,

    arXiv

  6. [18]

    MAMO: a mathematical modeling benchmark with solvers

    Huang, X., Shen, Q., Hu, Y., Gao, A., and Wang, B. MAMO: a mathematical modeling benchmark with solvers. arXiv:2405.13144, 2024b. Hutter, F., Hoos, H. H., and Leyton-Brown, K. Sequential model-based optimization for general algorithm configuration. In LION,

    arXiv

  7. [20]

    The evolutionary computation methods no one should use

    Kudela, J. The evolutionary computation methods no one should use. arXiv preprint arXiv:2301.01984,

    arXiv

  8. [21]

    Formalization of algorithms for optimization with block structures

    Li, C., Wang, Z., Bai, Y., Duan, Y., Gao, Y., Hao, P ., and Wen, Z. Formalization of algorithms for optimization with block structures. arXiv preprint arXiv:2503.18806, 2025a. Li, C., Xu, S., Sun, C., Zhou, L., and Wen, Z. Formalization of optimality conditions for smooth constrained optimization problems. arXiv preprint arXiv:2503.18821, 2025b. Li, K. an...

    arXiv

Show all 57 references
  1. [22]

    Synthesizing mixed-integer linear programming models from natural language descriptions

    Li, Q., Zhang, L., and Mak-Hau, V . Synthesizing mixed-integer linear programming models from natural language descriptions. arXiv:2311.15271,

    arXiv

  2. [23]

    Multi-agent credit assignment with pretrained language models

    Li, W., Qiao, D., Wang, B., Wang, X., Jin, B., and Zha, H. Multi-agent credit assignment with pretrained language models. In AISTATS, 2025c. Li, Z.-Z., Zhang, D., Zhang, M.-L., Zhang, J., Liu, Z., Yao, Y., Xu, H., Zheng, J., Wang, P .-J., Chen, X., et al. From system 1 to syst...

    arXiv

  3. [24]

    Algorithm evolution using large language model

    Liu, F., Tong, X., Yuan, M., and Zhang, Q. Algorithm evolution using large language model. arXiv:2311.15249,

    arXiv

  4. [25]

    Evolution of heuristics: Towards efficient automatic algorithm design using large language model

    22 Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows Liu, F., Tong, X., Yuan, M., Lin, X., Luo, F., Wang, Z., Lu, Z., and Zhang, Q. Evolution of heuristics: Towards efficient automatic algorithm design using large language model. In ICML, 2024a. Liu...

    arXiv

  5. [27]

    J., Liang, W., Wang, G., Huang, D.-A., Bastani, O., Jayaraman, D., Zhu, Y., Fan, L., and Anandkumar, A

    Ma, Y. J., Liang, W., Wang, G., Huang, D.-A., Bastani, O., Jayaraman, D., Zhu, Y., Fan, L., and Anandkumar, A. Eureka: Human-level reward design via coding large language models. In ICLR, 2024a. Ma, Z., Guo, H., Gong, Y.-J., Zhang, J., and Tan, K. C. Toward automated algorithm...

    arXiv

  6. [28]

    Sequential large language model-based hyper-parameter optimization

    Mahammadli, K. Sequential large language model-based hyper-parameter optimization. arXiv:2410.20302,

    arXiv

  7. [29]

    and Freitas, A

    Meadows, J. and Freitas, A. A survey in mathematical language processing.arXiv:2205.15231,

    arXiv

  8. [30]

    Gsm- symbolic: Understanding the limitations of mathematical reasoning in large language models

    Mirzadeh, I., Alizadeh, K., Shahrokhi, H., Tuzel, O., Bengio, S., and Farajtabar, M. Gsm- symbolic: Understanding the limitations of mathematical reasoning in large language models. arXiv:2410.05229,

    arXiv

  9. [32]

    T., He, S., Rengan, V ., Banitalebi-Dehkordi, A., Zhou, Z., and Zhang, Y

    Ramamonjison, R., Li, H., Yu, T. T., He, S., Rengan, V ., Banitalebi-Dehkordi, A., Zhou, Z., and Zhang, Y. Augmenting operations research with auto-formulation of optimization models from problem descriptions. arXiv:2209.15565,

    arXiv

  10. [33]

    NL4OPT competition: Formulating optimization problems based on their natural language descriptions

    Ramamonjison, R., Yu, T., Li, R., Li, H., Carenini, G., Ghaddar, B., He, S., Mostajabdaveh, M., Banitalebi-Dehkordi, A., Zhou, Z., et al. NL4OPT competition: Formulating optimization problems based on their natural language descriptions. In NeurIPS 2022 Competition Track,

    2022

  11. [34]

    E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., et al

    24 Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows Roziere, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., et al. Code Llama: Open foundation models for code. arXiv:2308.12950,

    arXiv

  12. [35]

    An overview of gradient descent optimization algorithms

    Ruder, S. An overview of gradient descent optimization algorithms. arXiv:1609.04747,

    arXiv

  13. [38]

    ORLM: Training large language models for optimization modeling

    Tang, Z., Huang, C., Zheng, X., Hu, S., Wang, Z., Ge, D., and Wang, B. ORLM: Training large language models for optimization modeling. arXiv:2405.17743,

    arXiv

  14. [39]

    Trirat, P ., Jeong, W., and Hwang, S. J. Automl-agent: A multi-agent llm framework for full-pipeline automl. arXiv:2410.02958,

    arXiv

  15. [40]

    Holy Grail 2.0: From natural language to constraint models

    Tsouros, D., Verhaeghe, H., Kadıo˘glu, S., and Guns, T. Holy Grail 2.0: From natural language to constraint models. arXiv:2308.01589,

    arXiv

  16. [41]

    Large language models still can’t plan (a benchmark for LLMs on planning and reasoning about change)

    Valmeekam, K., Olmo, A., Sreedharan, S., and Kambhampati, S. Large language models still can’t plan (a benchmark for LLMs on planning and reasoning about change). In NeurIPS 2022 Foundation Models for Decision Making Workshop,

    2022

  17. [42]

    LLMs still can’t plan; Can lrms? A preliminary evaluation of OpenAI’s o1 on planbench

    Valmeekam, K., Stechly, K., and Kambhampati, S. LLMs still can’t plan; Can lrms? A preliminary evaluation of OpenAI’s o1 on planbench. arXiv:2409.13373,

    arXiv

  18. [44]

    d., Mirzazadeh, F., Birbil, I., Kurtz, J., and Maragno, D

    Wasserkrug, S., Boussioux, L., Hertog, D. d., Mirzazadeh, F., Birbil, I., Kurtz, J., and Maragno, D. From large language models and optimization to decision optimization copilot: A research manifesto. arXiv:2402.16269,

    arXiv

  19. [45]

    ADMM penalty parameter selection by residual balancing

    Wohlberg, B. ADMM penalty parameter selection by residual balancing. arXiv:1704.06209,

    arXiv

  20. [47]

    The rise and potential of large language model based agents: A survey

    Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. The rise and potential of large language model based agents: A survey. arXiv:2309.07864,

    arXiv

  21. [48]

    Towards large reasoning models: A survey of reinforced reasoning with large language models

    Xu, F., Hao, Q., Zong, Z., Wang, J., Zhang, Y., Wang, J., Lan, X., Gong, J., Ouyang, T., Meng, F., et al. Towards large reasoning models: A survey of reinforced reasoning with large language models. arXiv preprint arXiv:2501.09686,

    arXiv

  22. [49]

    A survey on knowledge distillation of large language models

    Xu, X., Li, M., Tao, C., Shen, T., Cheng, R., Li, J., Xu, C., Tao, D., and Zhou, T. A survey on knowledge distillation of large language models. arXiv:2402.13116,

    arXiv

  23. [50]

    Admm without a fixed penalty parameter: Faster convergence with new adaptive penalization

    Xu, Y., Liu, M., Lin, Q., and Yang, T. Admm without a fixed penalty parameter: Faster convergence with new adaptive penalization. NeurIPS, 2017a. Xu, Z., Figueiredo, M., and Goldstein, T. Adaptive admm with spectral penalty parameter selection. In AISTATS, 2017b. Yadkori, Y. A...

    arXiv

  24. [51]

    Yang, Y., Guan, X., Jia, Q.-S., Yu, L., Xu, B., and Spanos, C. J. A survey of ADMM variants for distributed optimization: Problems, algorithms and features. arXiv:2208.03700,

    arXiv

  25. [52]

    Taking human out of learning applications: A survey on automated machine learning

    Yao, Q., Wang, M., Chen, Y., Dai, W., Li, Y.-F., Tu, W.-W., Yang, Q., and Yu, Y. Taking human out of learning applications: A survey on automated machine learning. arXiv:1810.13306,

    arXiv

  26. [53]

    React: Synergizing reasoning and acting in language models

    Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. React: Synergizing reasoning and acting in language models. arXiv:2210.03629,

    arXiv

  27. [54]

    Multi-objective evolution of heuristic using large language model

    Yao, S., Liu, F., Lin, X., Lu, Z., Wang, Z., and Zhang, Q. Multi-objective evolution of heuristic using large language model. arXiv:2409.16867, 2024a. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., and Narasimhan, K. Tree of thoughts: Deliberate problem solvin...

    arXiv

  28. [55]

    Using large language models for hyperparameter optimization

    Zhang, M., Desai, N., Bae, J., Lorraine, J., and Ba, J. Using large language models for hyperparameter optimization. In NeurIPS 2023 Foundation Models for Decision Making Workshop, 2023a. Zhang, R., Liu, F., Lin, X., Wang, Z., Lu, Z., and Zhang, Q. Understanding the importance...

    2023

  29. [56]

    AutoML-GPT: Automatic machine learning with GPT

    Zhang, S., Gong, C., Wu, L., Liu, X., and Zhou, M. AutoML-GPT: Automatic machine learning with GPT. arXiv:2305.02499, 2023b. Zheng, J., Qiu, S., Shi, C., and Ma, Q. Towards lifelong learning of large language models: A survey. ACM Computing Surveys, 57(8):1–35, 2025a. Zheng, J...

    arXiv

  30. [1965]

    A novel approach for auto-formulation of optimization problems

    Ning, Y., Liu, J., Qin, L., Xiao, T., Xue, S., Huang, Z., Liu, Q., Chen, E., and Wu, J. A novel approach for auto-formulation of optimization problems. arXiv:2302.04643,

    arXiv

  31. [1997]

    Wu, X., Wu, S.-h., Wu, J., Feng, L., and Tan, K. C. Evolutionary computation in the era of large language model: Survey and roadmap. arXiv:2401.10034,

    arXiv

  32. [1999]

    J., Taylor, C

    21 Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows Jiang, B., Xie, Y., Hao, Z., Wang, X., Mallick, T., Su, W. J., Taylor, C. J., and Roth, D. A peek into token bias: Large language models are not yet genuine reasoners. arXiv:2406.11050, 2024a. Jia...

    arXiv

  33. [2000]

    Large language models for constructing and optimizing machine learning workflows: A survey

    Gu, Y., You, H., Cao, J., and Yu, M. Large language models for constructing and optimizing machine learning workflows: A survey. arXiv:2411.10478,

    arXiv

  34. [2007]

    Constitutional ai: Harmlessness from ai feedback

    Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., et al. Constitutional ai: Harmlessness from ai feedback. arXiv:2212.08073,

    arXiv

  35. [2010]

    A survey of small language models

    Van Nguyen, C., Shen, X., Aponte, R., Xia, Y., Basu, S., Hu, Z., Chen, J., Parmar, M., Kunapuli, S., Barrow, J., et al. A survey of small language models. arXiv:2410.20011,

    arXiv

  36. [2011]

    Zhou, X., Tie, G., Zhang, G., Wang, W., Zuo, Z., Wu, D., Chu, D., Zhou, P ., Sun, L., and Gong, N. Z. Large reasoning models in agent scenarios: Exploring the necessity of reasoning capabilities. arXiv preprint arXiv:2503.11074,

    arXiv

  37. [2012]

    Towards large language models as copilots for theorem proving in lean

    Song, P ., Yang, K., and Anandkumar, A. Towards large language models as copilots for theorem proving in lean. arXiv:2404.12534,

    arXiv

  38. [2013]

    Optimus: Optimization modeling using mip solvers and large language models

    AhmadiTeshnizi, A., Gao, W., and Udell, M. Optimus: Optimization modeling using mip solvers and large language models. arXiv:2310.06116,

    arXiv

  39. [2015]

    Data interpreter: An LLM agent for data science

    Hong, S., Lin, Y., Liu, B., Liu, B., Wu, B., Zhang, C., Wei, C., Li, D., Chen, J., Zhang, J., et al. Data interpreter: An LLM agent for data science. arXiv:2402.18679,

    arXiv

  40. [2016]

    and Sch¨utze, H

    Schick, T. and Sch¨utze, H. It’s not just size that matters: Small language models are also few-shot learners. arXiv:2009.07118,

    2009 arXiv

  41. [2017]

    Exploiting LLM quantization

    Egashira, K., Vero, M., Staab, R., He, J., and Vechev, M. Exploiting LLM quantization. arXiv:2405.18137,

    arXiv

  42. [2018]

    Large language model agent for hyper-parameter optimization

    Liu, S., Gao, C., and Li, Y. Large language model agent for hyper-parameter optimization. arXiv:2402.01881, 2024d. Liu, X., Lou, X., Jiao, J., and Zhang, J. Position: Foundation agents as the paradigm shift for decision making. In ICML, 2024e. L´opez-Ib´a˜nez, M., Dubois-Lacos...

    arXiv

  43. [2019]

    Towards an automatic optimisation model generator assisted with generative pre-trained transformer

    Almonacid, B. Towards an automatic optimisation model generator assisted with generative pre-trained transformer. arXiv:2305.05811,

    arXiv

  44. [2020]

    Fireact: Toward language agent fine-tuning

    Chen, B., Shu, C., Shareghi, E., Collier, N., Narasimhan, K., and Yao, S. Fireact: Toward language agent fine-tuning. arXiv preprint arXiv:2310.05915, 2023a. Chen, H., Constante-Flores, G. E., and Li, C. Diagnosing infeasible optimization problems using large language models. ...

    arXiv

  45. [2021]

    When large language model meets optimization

    Huang, S., Yang, K., Qi, S., and Wang, R. When large language model meets optimization. arXiv:2405.10098, 2024a. Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P ., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y., Sermanet, P ., Jackson, T., Brown, N., Luu, L.,...

    arXiv

  46. [2022]

    Bartz-Beielstein, T., Doerr, C., Berg, D. v. d., Bossek, J., Chandrasekaran, S., Eftimov, T., Fischbach, A., Kerschke, P ., La Cava, W., Lopez-Ibanez, M., et al. Benchmarking in optimization: Best practice and open issues. arXiv:2007.03488,

    2007 arXiv

  47. [2023]

    T., Nguyen, S., Sun, Y., and Alahakoon, D

    Amarasinghe, P . T., Nguyen, S., Sun, Y., and Alahakoon, D. AI-Copilot for business op- timisation: A framework and a case study in production scheduling. arXiv:2309.13218,

    arXiv

  48. [2024]

    and Mattingley, J

    Boyd, S. and Mattingley, J. Branch and bound methods. Notes for EE364b, Stanford University, 2006:07,

    2006

  49. [2025]

    M., and Yu, Y

    Chen, X., Deng, T., Shen, Z.-J. M., and Yu, Y. Mind the gap between research and practice in operations management. IISE Transactions, 55(1):32–42, 2023b. 19 Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows Cheng, Y., Zhang, C., Zhang, Z., Meng, X....

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.