Trust-region filter algorithms utilizing Hessian information for gray-box optimization

Antonio del Rio Chanona; Gul Hameed; Lorenz T. Biegler; Michael Short; Tao Chen

arxiv: 2509.01651 · v4 · submitted 2025-09-01 · 🧮 math.OC

Trust-region filter algorithms utilizing Hessian information for gray-box optimization

Gul Hameed , Tao Chen , Antonio del Rio Chanona , Lorenz T. Biegler , Michael Short This is my paper

Pith reviewed 2026-05-18 19:22 UTC · model grok-4.3

classification 🧮 math.OC

keywords gray-box optimizationtrust-region methodsHessian approximationsurrogate-based optimizationfilter algorithmsblack-box optimizationprocess optimization

0 comments

The pith

Four Hessian-informed trust-region filter variants achieve order-of-magnitude reductions in iterations and black-box evaluations for gray-box optimization while requiring less tuning.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The authors develop four new versions of the trust-region filter algorithm that incorporate information from projected positive definite Hessians to automatically determine step sizes, greatly reducing the need for manual parameter tuning. These variants are tested with both simple linear or quadratic surrogates and more accurate Taylor or Gaussian process surrogates for the black-box parts of the model. Across 25 standard test problems and five real engineering examples, the methods cut the number of expensive black-box evaluations and total iterations substantially compared to the standard algorithm and also beat several derivative-free solvers. High-accuracy surrogates enabled solving nearly all problems, suggesting the approach makes gray-box optimization practical for complex industrial systems where evaluations are costly.

Core claim

By using projected positive definite Hessians to scale steps automatically within trust-region filter frameworks and approximating black-box components with surrogates of different fidelities, the new algorithms demonstrate substantial efficiency gains on gray-box problems, solving between 92 and 100 percent of cases with high-fidelity approximations and showing robustness to parameter choices.

What carries the argument

Projected positive definite Hessians that enable automatic step scaling in the four new trust-region filter variants.

If this is right

Automatic step scaling reduces reliance on extensive parameter tuning.
High-fidelity surrogates achieve solution rates of 92 to 100 percent on test problems.
Order-of-magnitude decreases occur in both iterations and black-box evaluations.
Performance exceeds that of classical derivative-free optimization methods.
These approaches provide more scalable solutions for industrial gray-box models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Integration of Hessian information could improve efficiency in other hybrid optimization settings that mix algebraic and simulation-based models.
Lower evaluation counts might support applications in online optimization where simulation time is limited.
Further comparisons with global optimization techniques could clarify performance on multimodal gray-box landscapes.
Extension to constrained problems with more complex black-box interactions remains an open direction suggested by the results.

Load-bearing premise

The assumption that projected positive definite Hessians can be reliably obtained without introducing instability or inaccurate models in gray-box optimization settings.

What would settle it

A set of tests on gray-box benchmarks in which the Hessian-informed variants require similar or greater numbers of black-box evaluations and iterations as the classical trust-region filter method.

Figures

Figures reproduced from arXiv: 2509.01651 by Antonio del Rio Chanona, Gul Hameed, Lorenz T. Biegler, Michael Short, Tao Chen.

read the original abstract

Optimizing industrial processes often involves gray-box models that couple algebraic glass-box equations with black-box components lacking analytic derivatives. Such systems challenge derivative-based solvers. The classical trust-region filter (TRF) algorithm provides a robust framework but requires extensive parameter tuning and numerous black-box evaluations. This work introduces four Hessian-informed TRF variants that use projected positive definite Hessians for automatic step scaling and minimal tuning, combined with both low-fidelity (linear, quadratic) and high-fidelity (Taylor series, Gaussian process) surrogates for local black-box approximation. Tested on 25 gray-box benchmarks and five engineering case studies, the new variants achieved up to order-of-magnitude reductions in iterations and black-box evaluations, with reduced sensitivity to tuning parameters relative to the classical TRF algorithm. High-fidelity surrogates solved 92%-100% of problems, compared with 72%-84% for low-fidelity surrogates. The developed TRF methods also outperformed classical derivative-free optimization solvers. Results show that new variants offer robust, scalable alternatives for gray-box optimization.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper adds four Hessian-informed trust-region filter variants that use projected positive definite Hessians for automatic step scaling and pair them with low- and high-fidelity surrogates, delivering clear reductions in iterations and evaluations on gray-box benchmarks.

read the letter

The main thing here is four new trust-region filter variants that bring in projected positive definite Hessians to scale steps automatically and cut down on tuning for gray-box problems that mix algebraic equations with black-box parts. They test these against both low-fidelity surrogates like linear and quadratic models and higher-fidelity ones like Taylor series and Gaussian processes. The empirical side is straightforward: on 25 benchmarks and five engineering cases the variants often need far fewer iterations and black-box calls than the classical TRF, sometimes by an order of magnitude, and they show less sensitivity to parameter choices. High-fidelity surrogates reach 92-100% solve rates while low-fidelity ones sit at 72-84%, and the methods also beat some standard derivative-free solvers. That practical improvement for process engineering work is the clearest contribution. The softer spot is the Hessian projection step itself. In true gray-box settings the black-box pieces lack analytic derivatives, so any Hessian comes from finite differences or the surrogate. The paper describes the projection to positive definite form, but it is not obvious how much protection exists against noise or poor curvature estimates that could make the local model unreliable. The benchmark results hold up on the problems they ran, yet a bit more on safeguards or failure cases would strengthen the robustness claim. This is for readers who work on optimization in chemical engineering or similar fields where models are partly known and partly simulated. Anyone who needs concrete algorithmic tweaks tested on relevant gray-box instances will get usable ideas from the comparisons. The work is concrete enough and the evidence is empirical rather than circular, so it deserves a serious referee even if the Hessian details could use extra scrutiny.

Referee Report

1 major / 2 minor

Summary. The paper introduces four Hessian-informed variants of the trust-region filter (TRF) algorithm for gray-box optimization problems that combine algebraic equations with black-box components. The variants employ projected positive definite Hessians for automatic step scaling with minimal tuning, paired with low-fidelity (linear/quadratic) and high-fidelity (Taylor series/Gaussian process) surrogates. Empirical tests on 25 gray-box benchmarks and five engineering case studies report up to order-of-magnitude reductions in iterations and black-box evaluations, higher solve rates (92-100% for high-fidelity vs. 72-84% for low-fidelity), reduced parameter sensitivity relative to classical TRF, and outperformance of derivative-free solvers.

Significance. If the performance claims hold, the work offers practical improvements for industrial gray-box optimization by reducing evaluation counts and tuning burden. The explicit empirical evaluation across 25 benchmarks plus five case studies provides concrete, falsifiable evidence of gains over baselines, which strengthens the contribution in the math.OC setting.

major comments (1)

[Methods / Algorithm description] The central claims of order-of-magnitude reductions, reduced tuning sensitivity, and 92-100% solve rates rest on the four Hessian-informed TRF variants. The manuscript provides no explicit description of the projection operator, the positive-definiteness correction, or safeguards against negative curvature/noise when Hessians are approximated from black-box components (see the description of the four variants and the step-scaling procedure). This is load-bearing for the robustness claims in gray-box settings.

minor comments (2)

[Abstract and Results] The abstract and results section would benefit from a brief table summarizing the four variants' key differences (e.g., Hessian handling and surrogate type) to improve readability.
[Numerical results] Clarify the exact definition of 'solve rate' and any exclusion criteria for the 25 benchmarks to allow direct replication of the reported percentages.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. We address the major comment below and will incorporate the requested clarifications in a revised version.

read point-by-point responses

Referee: The central claims of order-of-magnitude reductions, reduced tuning sensitivity, and 92-100% solve rates rest on the four Hessian-informed TRF variants. The manuscript provides no explicit description of the projection operator, the positive-definiteness correction, or safeguards against negative curvature/noise when Hessians are approximated from black-box components (see the description of the four variants and the step-scaling procedure). This is load-bearing for the robustness claims in gray-box settings.

Authors: We agree that the manuscript would benefit from a more explicit and self-contained description of these algorithmic components to support reproducibility and the robustness claims. In the revised manuscript we will expand the section detailing the four variants and the step-scaling procedure to include: (i) the precise definition of the projection operator onto the positive-definite cone, (ii) the specific positive-definiteness correction (e.g., eigenvalue shifting or modified Cholesky factorization), and (iii) any regularization or noise-handling safeguards applied when Hessians are obtained from black-box evaluations. These additions will be presented with the relevant mathematical expressions and pseudocode while preserving the original algorithmic structure and empirical results. revision: yes

Circularity Check

0 steps flagged

No circularity: algorithmic modifications validated by external benchmarks

full rationale

The paper proposes four new Hessian-informed trust-region filter variants that incorporate projected positive definite Hessians for step scaling and combine them with low- and high-fidelity surrogates. These are presented as explicit algorithmic changes to the classical TRF method. All performance claims (iteration reductions, solve rates, reduced tuning sensitivity) rest on direct testing against 25 gray-box benchmarks and five engineering case studies, not on any internal derivation, fitted parameter renamed as prediction, or self-referential equation. References to prior TRF work serve only as baseline context and do not carry the central claims. No load-bearing step reduces to its own inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claims rest on standard assumptions from trust-region and surrogate modeling literature plus the empirical behavior observed on the chosen benchmarks; no new free parameters or invented entities are introduced in the abstract.

axioms (1)

domain assumption Local approximations of black-box components via linear, quadratic, Taylor, or Gaussian process surrogates remain sufficiently accurate within trust regions to guide reliable steps.
This underpins the use of both low- and high-fidelity surrogates in the new variants.

pith-pipeline@v0.9.0 · 5723 in / 1318 out tokens · 53622 ms · 2026-05-18T19:22:27.007433+00:00 · methodology

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

A trust-region funnel algorithm for gray-box optimization
math.OC 2025-11 unverdicted novelty 5.0

A trust-region funnel algorithm for gray-box optimization achieves global convergence to first-order critical points and performs comparably or better than the classical trust-region filter method.

Reference graph

Works this paper leans on

77 extracted references · 77 canonical work pages · cited by 1 Pith paper

[1]

A trust region filter method for glass box/black box optimization,

J. P. Eason and L. T. Biegler, “A trust region filter method for glass box/black box optimization,” AIChE Journal, vol. 62, no. 9, pp. 3124–3136, 2016

work page 2016
[2]

CONOPT: A GRG code for large sparse dynamic nonlinear optimization problems,

A. Drud, “CONOPT: A GRG code for large sparse dynamic nonlinear optimization problems,” Math Program, vol. 31, no. 2, pp. 153–191, 1985, doi: 10.1007/BF02591747

work page doi:10.1007/bf02591747 1985
[3]

B. A. Murtagh and M. A. Saunders, MINOS 5.0 user’s guide , vol. 83, no. 20. Systems Optimization Laboratory, Department of Operations Research, Stanford …, 1983

work page 1983
[4]

SIAM Review 47, 99–131

P. E. Gill, W. Murray, and M. A. Saunders, “SNOPT: An SQP Algorithm for Large -Scale Constrained Optimization,” SIAM Review , vol. 47, no. 1, pp. 99 –131, 2005, doi: 10.1137/S0036144504446096

work page doi:10.1137/s0036144504446096 2005
[5]

KNITRO: An Integrated Package for Nonlinear Optimization,

R. H. Byrd, J. Nocedal, and R. A. Waltz, “KNITRO: An Integrated Package for Nonlinear Optimization,” in Large-Scale Nonlinear Optimization , G. Di Pillo and M. Roma, Eds., Boston, MA: Springer US, 2006, pp. 35–59. doi: 10.1007/0-387-30065-1_4

work page doi:10.1007/0-387-30065-1_4 2006
[6]

LOQO:an interior point code for quadratic programming,

R. J. Vanderbei, “LOQO:an interior point code for quadratic programming,” Optim Methods Softw, vol. 11, no. 1–4, pp. 451–484, Jan. 1999, doi: 10.1080/10556789908805759

work page doi:10.1080/10556789908805759 1999
[7]

On the implementation of an interior-point ﬁlter line-search algorithm for large-scale nonlinear programming,

A. Wächter and L. T. Biegler, “On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming,” Math Program, vol. 106, no. 1, pp. 25– 57, 2006, doi: 10.1007/s10107-004-0559-y. Page 31 of 40

work page doi:10.1007/s10107-004-0559-y 2006
[8]

Error estimation for surrogate models with noisy small -sized training sets,

J. Wackers, H. P. Solak, R. Pellegrini, A. Serani, and M. Diez, “Error estimation for surrogate models with noisy small -sized training sets,” in Adaptive Modelling and Simulation (ADMOS 2023), 2023

work page 2023
[9]

A. R. Conn, K. Scheinberg, and L. N. Vicente, Introduction to Derivative -Free Optimization. Society for Industrial and Applied Mathematics, 2009. doi: 10.1137/1.9780898718768

work page doi:10.1137/1.9780898718768 2009
[10]

Advanced trust region optimization strategies for glass box/black box models,

J. P. Eason and L. T. Biegler, “Advanced trust region optimization strategies for glass box/black box models,” AIChE Journal, vol. 64, no. 11, pp. 3934–3943, 2018

work page 2018
[11]

Simulation -based optimization of distillation processes using an extended cutting plane algorithm,

J. Javaloyes-Antón, J. Kronqvist, and J. A. Caballero, “Simulation -based optimization of distillation processes using an extended cutting plane algorithm,” Comput Chem Eng, vol. 159, p. 107655, 2022, doi: https://doi.org/10.1016/j.compchemeng.2021.107655

work page doi:10.1016/j.compchemeng.2021.107655 2022
[12]

Derivative -free optimization methods,

J. Larson, M. Menickelly, and S. M. Wild, “Derivative -free optimization methods,” Acta Numerica, vol. 28, pp. 287–404, 2019, doi: DOI: 10.1017/S0962492919000060

work page doi:10.1017/s0962492919000060 2019
[13]

A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation,

M. J. D. Powell, “A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation,” in Advances in Optimization and Numerical Analysis, S. Gomez and J.-P. Hennart, Eds., Dordrecht: Springer Netherlands, 1994, pp. 51–

work page 1994
[14]

doi: 10.1007/978-94-015-8330-5_4

work page doi:10.1007/978-94-015-8330-5_4
[15]

The BOBYQA algorithm for bound constrained optimization without derivatives,

M. J. D. Powell, “The BOBYQA algorithm for bound constrained optimization without derivatives,” Cambridge NA Report NA2009/06, University of Cambridge, Cambridge, vol. 26, pp. 26–46, 2009

work page 2009
[16]

UOBYQA: unconstrained optimization by quadratic approximation,

M. J. D. Powell, “UOBYQA: unconstrained optimization by quadratic approximation,” Math Program, vol. 92, no. 3, pp. 555–582, 2002, doi: 10.1007/s101070100290

work page doi:10.1007/s101070100290 2002
[17]

The NEWUOA software for unconstrained optimization without derivatives,

M. J. D. Powell, “The NEWUOA software for unconstrained optimization without derivatives,” in Large-Scale Nonlinear Optimization , G. Di Pillo and M. Roma, Eds., Boston, MA: Springer US, 2006, pp. 255–297. doi: 10.1007/0-387-30065-1_16

work page doi:10.1007/0-387-30065-1_16 2006
[18]

Le Digabel, C

S. Le Digabel, C. Tribes, and C. Audet, NOMAD User Guide: Version 3.6. 0. GERAD HEC Montréal, 2013

work page 2013
[19]

Dakota 6.22.0 documentation,

B. M. Adams et al. , “Dakota 6.22.0 documentation,” Albuquerque, NM, Nov. 2024. Accessed: Jul. 01, 2025. [Online]. Available: https://snl-dakota.github.io/

work page 2024
[20]

Model-Based Derivative-Free Optimization Methods andSoftware,

T. M. Ragonneau, “Model-based derivative-free optimization methods and software,” arXiv preprint arXiv:2210.12018, 2022

work page arXiv 2022
[21]

Derivative -free optimization: a review of algorithms and comparison of software implementations,

L. M. Rios and N. V Sahinidis, “Derivative -free optimization: a review of algorithms and comparison of software implementations,” Journal of Global Optimization, vol. 56, no. 3, pp. 1247–1293, 2013, doi: 10.1007/s10898-012-9951-y

work page doi:10.1007/s10898-012-9951-y 2013
[22]

A Trust Region Filter Algorithm for Surrogate-based Optimization,

J. P. Eason, “A Trust Region Filter Algorithm for Surrogate-based Optimization,” Doctoral Dissertation, Carnegie Mellon University, Feb. 2018, doi: 10.1184/R1/6714431.v1

work page doi:10.1184/r1/6714431.v1 2018
[23]

A. R. Conn, N. I. M. Gould, and P. L. Toint, Trust Region Methods. Society for Industrial and Applied Mathematics, 2000. doi: 10.1137/1.9780898719857

work page doi:10.1137/1.9780898719857 2000
[24]

Nonlinear programming without a penalty function,

R. Fletcher and S. Leyffer, “Nonlinear programming without a penalty function,” Math Program, vol. 91, no. 2, pp. 239–269, 2002, doi: 10.1007/s101070100244. Page 32 of 40

work page doi:10.1007/s101070100244 2002
[25]

A NOVEL FILTER TRUST-REGION ALGORITHM FOR CONSTRAINED OPTIMIZATION USING REDUCED ORDER MODELING,

A. Agarwal and L. T. Biegler, “A NOVEL FILTER TRUST-REGION ALGORITHM FOR CONSTRAINED OPTIMIZATION USING REDUCED ORDER MODELING,” Chemical Engineering Department, Carnegie Mellon University, Accessed: Jan. 29, 2024. [Online]. Available: http://focapo.cheme.cmu.edu/2012/proceedings/data/papers/040.pdf

work page 2024
[26]

A trust-region framework for constrained optimization using reduced order modeling,

A. Agarwal and L. T. Biegler, “A trust-region framework for constrained optimization using reduced order modeling,” Optimization and Engineering , vol. 14, no. 1, pp. 3 –35, 2013, doi: 10.1007/s11081-011-9164-0

work page doi:10.1007/s11081-011-9164-0 2013
[27]

On an inexact trust -region SQP-filter method for constrained nonlinear optimization,

A. Walther and L. Biegler, “On an inexact trust -region SQP-filter method for constrained nonlinear optimization,” Comput Optim Appl , vol. 63, no. 3, pp. 613 –638, 2016, doi: 10.1007/s10589-015-9793-x

work page doi:10.1007/s10589-015-9793-x 2016
[28]

Demand‐based optimization of a chlorobenzene process with high‐fidelity and surrogate reactor models under trust region strategies,

N. Yoshio and L. T. Biegler, “Demand‐based optimization of a chlorobenzene process with high‐fidelity and surrogate reactor models under trust region strategies,” AIChE Journal, vol. 67, no. 1, p. e17054, 2021

work page 2021
[29]

Optimization strategies for produced water networks with integrated desalination facilities,

S. Naik, M. Zamarripa, M. Drouven, and L. T. Biegler, “Optimization strategies for produced water networks with integrated desalination facilities,” Comput Chem Eng, vol. 187, p. 108738, 2024, doi: https://doi.org/10.1016/j.compchemeng.2024.108738

work page doi:10.1016/j.compchemeng.2024.108738 2024
[30]

Efficient Trust Region Filter Modeling Strategies for Computationally Expensive Black -Box Optimization,

R. Liang, Y. Han, H. Hu, B. Chen, Z. Yuan, and L. T. Biegler, “Efficient Trust Region Filter Modeling Strategies for Computationally Expensive Black -Box Optimization,” Comput Chem Eng, p. 108816, 2024, doi: https://doi.org/10.1016/j.compchemeng.2024.108816

work page doi:10.1016/j.compchemeng.2024.108816 2024
[31]

The trust region filter strategy: Survey of a rigorous approach for optimization with surrogate models,

L. T. Biegler, “The trust region filter strategy: Survey of a rigorous approach for optimization with surrogate models,” Digital Chemical Engineering , vol. 13, p. 100197, 2024, doi: https://doi.org/10.1016/j.dche.2024.100197

work page doi:10.1016/j.dche.2024.100197 2024
[32]

Graduate Texts in Mathematics, vol 67

M. G. Ierapetritou and E. N. Pistikopoulos, “Global Optimization for Stochastic Planning, Scheduling and Design Problems,” in Global Optimization in Engineering Design , I. E. Grossmann, Ed., Boston, MA: Springer US, 1996, pp. 231–287. doi: 10.1007/978-1-4757- 5331-8_8

work page doi:10.1007/978-1-4757- 1996
[33]

Hybrid simulation based optimization approach for supply chain management,

A. Nikolopoulou and M. G. Ierapetritou, “Hybrid simulation based optimization approach for supply chain management,” Comput Chem Eng , vol. 47, pp. 183 –193, 2012, doi: https://doi.org/10.1016/j.compchemeng.2012.06.045

work page doi:10.1016/j.compchemeng.2012.06.045 2012
[34]

Chapter 5 - Data-driven optimization algorithms,

B. Beykal and E. N. Pistikopoulos, “Chapter 5 - Data-driven optimization algorithms,” in Artificial Intelligence in Manufacturing , M. Soroush and R. D Braatz, Eds., Academic Press, 2024, pp. 135–180. doi: https://doi.org/10.1016/B978-0-323-99134-6.00006-2

work page doi:10.1016/b978-0-323-99134-6.00006-2 2024
[35]

Constrained optimization of black -box stochastic systems using a novel feasibility enhanced Kriging -based method,

Z. Wang and M. Ierapetritou, “Constrained optimization of black -box stochastic systems using a novel feasibility enhanced Kriging -based method,” Comput Chem Eng , vol. 118, pp. 210–223, 2018, doi: https://doi.org/10.1016/j.compchemeng.2018.07.016

work page doi:10.1016/j.compchemeng.2018.07.016 2018
[36]

ARGONAUT: AlgoRithms for Global Optimization of coNstrAined grey-box compUTational problems,

F. Boukouvala and C. A. Floudas, “ARGONAUT: AlgoRithms for Global Optimization of coNstrAined grey-box compUTational problems,” Optim Lett, vol. 11, no. 5, pp. 895–913, 2017, doi: 10.1007/s11590-016-1028-2

work page doi:10.1007/s11590-016-1028-2 2017
[37]

Global optimization of grey-box computational systems using surrogate functions and application Page 33 of 40 to highly constrained oil-field operations,

B. Beykal, F. Boukouvala, C. A. Floudas, N. Sorek, H. Zalavadia, and E. Gildin, “Global optimization of grey-box computational systems using surrogate functions and application Page 33 of 40 to highly constrained oil-field operations,” Comput Chem Eng, vol. 114, pp. 99–110, 2018, doi: https://doi.org/10.1016/j.compchemeng.2018.01.005

work page doi:10.1016/j.compchemeng.2018.01.005 2018
[38]

Learning surrogate models for simulation - based optimization,

A. Cozad, N. V Sahinidis, and D. C. Miller, “Learning surrogate models for simulation - based optimization,” AIChE Journal , vol. 60, no. 6, pp. 2211 –2227, Jun. 2014, doi: https://doi.org/10.1002/aic.14418

work page doi:10.1002/aic.14418 2014
[39]

The ALAMO approach to machine learning,

Z. T. Wilson and N. V Sahinidis, “The ALAMO approach to machine learning,” Comput Chem Eng , vol. 106, pp. 785 –795, 2017, doi: https://doi.org/10.1016/j.compchemeng.2017.02.010

work page doi:10.1016/j.compchemeng.2017.02.010 2017
[40]

Deterministic global derivative -free optimization of black - box problems with bounded Hessian,

I. Bajaj and M. M. F. Hasan, “Deterministic global derivative -free optimization of black - box problems with bounded Hessian,” Optim Lett, vol. 14, no. 4, pp. 1011–1026, 2020, doi: 10.1007/s11590-019-01421-0

work page doi:10.1007/s11590-019-01421-0 2020
[41]

A trust region -based two phase algorithm for constrained black-box and grey-box optimization with infeasible initial point,

I. Bajaj, S. S. Iyer, and M. M. Faruque Hasan, “A trust region -based two phase algorithm for constrained black-box and grey-box optimization with infeasible initial point,” Comput Chem Eng , vol. 116, pp. 306 –321, 2018, doi: https://doi.org/10.1016/j.compchemeng.2017.12.011

work page doi:10.1016/j.compchemeng.2017.12.011 2018
[42]

ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions,

S. M. Wild, R. G. Regis, and C. A. Shoemaker, “ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions,” SIAM Journal on Scientific Computing , vol. 30, no. 6, pp. 3197–3219, 2008, doi: 10.1137/070691814

work page doi:10.1137/070691814 2008
[43]

Trust regions in Kriging -based optimization with expected improvement,

R. G. Regis, “Trust regions in Kriging -based optimization with expected improvement,” Engineering Optimization , vol. 48, no. 6, pp. 1037 –1059, Jun. 2016, doi: 10.1080/0305215X.2015.1082350

work page doi:10.1080/0305215x.2015.1082350 2016
[44]

Inexact Restoration method for nonlinear optimization without derivatives,

Ma. B. Arouxét, N. E. Echebest, and E. A. Pilotta, “Inexact Restoration method for nonlinear optimization without derivatives,” J Comput Appl Math , vol. 290, pp. 26 –43, 2015, doi: https://doi.org/10.1016/j.cam.2015.04.047

work page doi:10.1016/j.cam.2015.04.047 2015
[45]

Advances in surrogate based modeling, feasibility analysis, and optimization: A review,

A. Bhosekar and M. Ierapetritou, “Advances in surrogate based modeling, feasibility analysis, and optimization: A review,” Comput Chem Eng , vol. 108, pp. 250 –267, 2018, doi: https://doi.org/10.1016/j.compchemeng.2017.09.017

work page doi:10.1016/j.compchemeng.2017.09.017 2018
[46]

Neufang, E

M. Neufang, E. Pajak, D. van de Berg, Y. S. Lee, and E. A. del Rio Chanona, Surrogate- Based Optimization Techniques for Process Systems Engineering. December, 2024

work page 2024
[47]

Surrogate model optimization: a comparison case study with pooling problems of CO2 point sources,

H. A. Pedrozo, M. A. Zamarripa, A. Uribe-Rodríguez, G. Panagakos, M. S. Diaz, and L. T. Biegler, “Surrogate model optimization: a comparison case study with pooling problems of CO2 point sources,” Comput Chem Eng , vol. 200, p. 109199, 2025, doi: https://doi.org/10.1016/j.compchemeng.2025.109199

work page doi:10.1016/j.compchemeng.2025.109199 2025
[48]

Synthesis of Combined Heat and Mass Exchange Networks Via a Trust Region Filter Optimisation Algorithm Including Detailed Unit Designs,

S. R. Kazi, M. Short, and L. T. Biegler, “Synthesis of Combined Heat and Mass Exchange Networks Via a Trust Region Filter Optimisation Algorithm Including Detailed Unit Designs,” in Computer Aided Chemical Engineering, vol. 50, M. Türkay and R. Gani, Eds., Elsevier, 2021, pp. 13–18. doi: https://doi.org/10.1016/B978-0-323-88506-5.50003-6

work page doi:10.1016/b978-0-323-88506-5.50003-6 2021
[49]

A trust -region framework for managing the use of approximation models in optimization,

N. M. Alexandrov, J. E. Dennis, R. M. Lewis, and V. Torczon, “A trust -region framework for managing the use of approximation models in optimization,” Structural optimization, vol. 15, no. 1, pp. 16–23, 1998, doi: 10.1007/BF01197433. Page 34 of 40

work page doi:10.1007/bf01197433 1998
[50]

Benchmarking optimization software with COPS 3.0.,

E. D. Dolan, J. J. Moré, and T. S. Munson, “Benchmarking optimization software with COPS 3.0.,” Argonne National Lab., Argonne, IL (US), 2004

work page 2004
[51]

Stabler Neo -Hookean Simulation: Absolute Eigenvalue Filtering for Projected Newton,

H. Chen, H.-T. D. Liu, D. I. W. Levin, C. Zheng, and A. Jacobson, “Stabler Neo -Hookean Simulation: Absolute Eigenvalue Filtering for Projected Newton,” in ACM SIGGRAPH 2024 Conference Papers , in SIGGRAPH ’24. New York, NY, USA: Association for Computing Machinery, 2024. doi: 10.1145/3641519.3657433

work page doi:10.1145/3641519.3657433 2024
[52]

Trust -Region Eigenvalue Filtering for Projected Newton,

H. Chen, H. -T. D. Liu, A. Jacobson, D. I. W. Levin, and C. Zheng, “Trust -Region Eigenvalue Filtering for Projected Newton,” in SIGGRAPH Asia 2024 Conference Papers, in SA ’24. New York, NY, USA: Association for Computing Machinery, 2024. doi: 10.1145/3680528.3687650

work page doi:10.1145/3680528.3687650 2024
[53]

Taking the human out of the loop: A review of Bayesian optimization,

B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. De Freitas, “Taking the human out of the loop: A review of Bayesian optimization,” Proceedings of the IEEE, vol. 104, no. 1, pp. 148–175, 2016, doi: 10.1109/JPROC.2015.2494218

work page doi:10.1109/jproc.2015.2494218 2016
[54]

When Gaussian Process Meets Big Data: A Review of Scalable GPs,

H. Liu, Y.-S. Ong, X. Shen, and J. Cai, “When Gaussian Process Meets Big Data: A Review of Scalable GPs,” IEEE Trans Neural Netw Learn Syst , vol. 31, no. 11, pp. 4405 –4423, 2020, doi: 10.1109/TNNLS.2019.2957109

work page doi:10.1109/tnnls.2019.2957109 2020
[55]

C. K. I. Williams and C. E. Rasmussen, Gaussian processes for machine learning , vol. 2, no. 3. MIT press Cambridge, MA, 2006

work page 2006
[56]

Determination of eigenvectors with Lagrange multipliers,

W. Han, D. -W. Jung, J. Lee, and C. Yu, “Determination of eigenvectors with Lagrange multipliers,” Journal of the Korean Physical Society, vol. 78, no. 11, pp. 1018–1022, 2021, doi: 10.1007/s40042-021-00112-3

work page doi:10.1007/s40042-021-00112-3 2021
[57]

Generating Eigenvalue Bounds Using Optimization,

H. Wolkowicz, “Generating Eigenvalue Bounds Using Optimization,” in Nonlinear Analysis and Variational Problems: In Honor of George Isac, P. M. Pardalos, T. M. Rassias, and A. A. Khan, Eds., New York, NY: Springer New York, 2010, pp. 465 –490. doi: 10.1007/978-1-4419-0158-3_29

work page doi:10.1007/978-1-4419-0158-3_29 2010
[58]

Pyomo: modeling and solving mathematical programs in Python,

W. E. Hart, J.-P. Watson, and D. L. Woodruff, “Pyomo: modeling and solving mathematical programs in Python,” Math Program Comput , vol. 3, no. 3, pp. 219 –260, 2011, doi: 10.1007/s12532-011-0026-8

work page doi:10.1007/s12532-011-0026-8 2011
[59]

A survey of the trust region subproblem within a semidefinite framework,

C. Fortin, “A survey of the trust region subproblem within a semidefinite framework,” University of Waterloo, Waterloo, Ontario, Canada, 2000. Accessed: May 15, 2025. [Online]. Available: http://hdl.handle.net/10012/1038

work page 2000
[60]

A shifted Steihaug -Toint method for computing a trust-region step,

L. Lukšan, C. Matonoha, and J. Vlcek, “A shifted Steihaug -Toint method for computing a trust-region step,” Technická zpráva, vol. 914, 2004

work page 2004
[61]

Convergence conditions for ascent methods,

P. Wolfe, “Convergence conditions for ascent methods,” SIAM review, vol. 11, no. 2, pp. 226–235, 1969

work page 1969
[62]

Diagonally Perturbing a Symmetric Matrix to Make It Positive Definite,

N. Higham, “Diagonally Perturbing a Symmetric Matrix to Make It Positive Definite,” Research. Accessed: May 15, 2025. [Online]. Available: https://nhigham.com/2021/02/16/diagonally-perturbing-a-symmetric-matrix-to-make-it- positive-definite/?utm_source=chatgpt.com

work page 2025
[63]

Trust -region algorithms for training responses: machine learning methods using indefinite Hessian approximations,

J. B. Erway, G. Joshua, M. Roummel F., and R. and Omheni, “Trust -region algorithms for training responses: machine learning methods using indefinite Hessian approximations,” Page 35 of 40 Optim Methods Softw , vol. 35, no. 3, pp. 460 –487, May 2020, doi: 10.1080/10556788.2019.1624747

work page doi:10.1080/10556788.2019.1624747 2020
[64]

Robust quasistatic finite elements and flesh simulation,

J. Teran, E. Sifakis, G. Irving, and R. Fedkiw, “Robust quasistatic finite elements and flesh simulation,” in Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, 2005, pp. 181–190

work page 2005
[65]

R. T. Rockafellar, Convex analysis, vol. 28. Princeton university press, 1997

work page 1997
[66]

Numerical Optimization

J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed. New York, NY: Springer New York, NY, 2006. doi: https://doi.org/10.1007/978-0-387-40065-5

work page doi:10.1007/978-0-387-40065-5 2006
[67]

gulhameed361/TRF -Solver: TRF -Solver v0.0.0,

G. Hameed, “gulhameed361/TRF -Solver: TRF -Solver v0.0.0,” Nov. 2025, Zenodo. doi: 10.5281/zenodo.17639336

work page doi:10.5281/zenodo.17639336 2025
[68]

Virtual Library of Simulation Experiments: Test Functions and Datasets

S. Surjanovic and D. Bingham, “Virtual Library of Simulation Experiments: Test Functions and Datasets.” Accessed: Oct. 28, 2025. [Online]. Available: https://www.sfu.ca/~ssurjano/index.html

work page 2025
[69]

COBALT: COnstrained Bayesian optimizAtion of computationaLly expensive grey-box models exploiting derivaTive information,

J. A. Paulson and C. Lu, “COBALT: COnstrained Bayesian optimizAtion of computationaLly expensive grey-box models exploiting derivaTive information,” Comput Chem Eng , vol. 160, p. 107700, 2022, doi: https://doi.org/10.1016/j.compchemeng.2022.107700

work page doi:10.1016/j.compchemeng.2022.107700 2022
[70]

Yarin Gal, Riashat Islam, and Zoubin Ghahramani

E. D. Dolan and J. J. Moré, “Benchmarking optimization software with performance profiles,” Math Program, vol. 91, no. 2, pp. 201–213, 2002, doi: 10.1007/s101070100263

work page doi:10.1007/s101070100263 2002
[71]

A penalty -based algorithm proposal for engineering optimization problems,

G. Z. Oztas and S. Erdem, “A penalty -based algorithm proposal for engineering optimization problems,” Neural Comput Appl , vol. 35, no. 10, pp. 7635 –7658, 2023, doi: 10.1007/s00521-022-08058-8

work page doi:10.1007/s00521-022-08058-8 2023
[72]

T. F. Edgar, D. M. Himmelblau, and L. S. Lasdon, Optimization of Chemical Processes , 2nd ed. McGraw-Hill Chemical Engineering Series, 2001

work page 2001
[73]

A Benchmark of PDF Information Extraction Tools Using a Multi-task and Multi- domain Evaluation Framework for Academic Documents,

A. H. Gandomi and X. -S. Yang, “Benchmark Problems in Structural Optimization,” in Computational Optimization, Methods and Algorithms , S. Koziel and X. -S. Yang, Eds., Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 259 –281. doi: 10.1007/978 -3- 642-20859-1_12

work page doi:10.1007/978 2011
[74]

L. T. Biegler, Nonlinear programming: concepts, algorithms, and applications to chemical processes. Philadelphia: SIAM, 2010

work page 2010
[75]

GitHub Repository: Pyomo,

N. Yoshio, J. Eason, and L. T. Biegler, “GitHub Repository: Pyomo,” Trust Region Framework Method Solver. Accessed: Oct. 27, 2025. [Online]. Available: https://pyomo.readthedocs.io/en/6.8.1/explanation/solvers/trustregion.html Appendix A: Global Convergence Eason and Biegler established that the sequence of iterates generated by the classical TRF algorith...

work page 2025
[76]

If 𝜃(𝑘+1)≤ (1−𝛾𝜃)𝜃(𝑘) for all 𝑘, then 𝜃(𝑘)→ 0 for 𝑘→∞

work page
[77]

Summing from 𝑘= 𝑘′ to 𝑘=𝑘′+𝑁 gives ∑ (𝑓(𝑘)−𝑓(𝑘+1)) 𝑘′+𝑁 𝑘=𝑘′ ≥ 𝛾𝑓 ∑ 𝜃(𝑘) 𝑘′+𝑁 𝑘=𝑘′

Otherwise, there exists an infinite subsequence 𝑘′ such that 𝑓(𝑘)−𝑓(𝑘+1)≥ 𝛾𝑓𝜃(𝑘) for all 𝑘≥ 𝑘′. Summing from 𝑘= 𝑘′ to 𝑘=𝑘′+𝑁 gives ∑ (𝑓(𝑘)−𝑓(𝑘+1)) 𝑘′+𝑁 𝑘=𝑘′ ≥ 𝛾𝑓 ∑ 𝜃(𝑘) 𝑘′+𝑁 𝑘=𝑘′ . Thus, 𝑓(𝑘′)−𝑓(𝑘′+𝑁+1) ≥ 𝛾𝑓 ∑ 𝜃(𝑘) 𝑘′+𝑁 𝑘=𝑘′ . Since 𝑓 is bounded below and monotonically decreasing, ∑ 𝜃(𝑘)𝑘′+𝑁 𝑘=𝑘′ < ∞ for 𝑁→ ∞, implying 𝜃(𝑘)→0 as 𝑘→∞. Therefore, 𝑙𝑖𝑚 𝑘→∞ 𝜃(...

work page 2000

[1] [1]

A trust region filter method for glass box/black box optimization,

J. P. Eason and L. T. Biegler, “A trust region filter method for glass box/black box optimization,” AIChE Journal, vol. 62, no. 9, pp. 3124–3136, 2016

work page 2016

[2] [2]

CONOPT: A GRG code for large sparse dynamic nonlinear optimization problems,

A. Drud, “CONOPT: A GRG code for large sparse dynamic nonlinear optimization problems,” Math Program, vol. 31, no. 2, pp. 153–191, 1985, doi: 10.1007/BF02591747

work page doi:10.1007/bf02591747 1985

[3] [3]

B. A. Murtagh and M. A. Saunders, MINOS 5.0 user’s guide , vol. 83, no. 20. Systems Optimization Laboratory, Department of Operations Research, Stanford …, 1983

work page 1983

[4] [4]

SIAM Review 47, 99–131

P. E. Gill, W. Murray, and M. A. Saunders, “SNOPT: An SQP Algorithm for Large -Scale Constrained Optimization,” SIAM Review , vol. 47, no. 1, pp. 99 –131, 2005, doi: 10.1137/S0036144504446096

work page doi:10.1137/s0036144504446096 2005

[5] [5]

KNITRO: An Integrated Package for Nonlinear Optimization,

R. H. Byrd, J. Nocedal, and R. A. Waltz, “KNITRO: An Integrated Package for Nonlinear Optimization,” in Large-Scale Nonlinear Optimization , G. Di Pillo and M. Roma, Eds., Boston, MA: Springer US, 2006, pp. 35–59. doi: 10.1007/0-387-30065-1_4

work page doi:10.1007/0-387-30065-1_4 2006

[6] [6]

LOQO:an interior point code for quadratic programming,

R. J. Vanderbei, “LOQO:an interior point code for quadratic programming,” Optim Methods Softw, vol. 11, no. 1–4, pp. 451–484, Jan. 1999, doi: 10.1080/10556789908805759

work page doi:10.1080/10556789908805759 1999

[7] [7]

On the implementation of an interior-point ﬁlter line-search algorithm for large-scale nonlinear programming,

A. Wächter and L. T. Biegler, “On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming,” Math Program, vol. 106, no. 1, pp. 25– 57, 2006, doi: 10.1007/s10107-004-0559-y. Page 31 of 40

work page doi:10.1007/s10107-004-0559-y 2006

[8] [8]

Error estimation for surrogate models with noisy small -sized training sets,

J. Wackers, H. P. Solak, R. Pellegrini, A. Serani, and M. Diez, “Error estimation for surrogate models with noisy small -sized training sets,” in Adaptive Modelling and Simulation (ADMOS 2023), 2023

work page 2023

[9] [9]

A. R. Conn, K. Scheinberg, and L. N. Vicente, Introduction to Derivative -Free Optimization. Society for Industrial and Applied Mathematics, 2009. doi: 10.1137/1.9780898718768

work page doi:10.1137/1.9780898718768 2009

[10] [10]

Advanced trust region optimization strategies for glass box/black box models,

J. P. Eason and L. T. Biegler, “Advanced trust region optimization strategies for glass box/black box models,” AIChE Journal, vol. 64, no. 11, pp. 3934–3943, 2018

work page 2018

[11] [11]

Simulation -based optimization of distillation processes using an extended cutting plane algorithm,

J. Javaloyes-Antón, J. Kronqvist, and J. A. Caballero, “Simulation -based optimization of distillation processes using an extended cutting plane algorithm,” Comput Chem Eng, vol. 159, p. 107655, 2022, doi: https://doi.org/10.1016/j.compchemeng.2021.107655

work page doi:10.1016/j.compchemeng.2021.107655 2022

[12] [12]

Derivative -free optimization methods,

J. Larson, M. Menickelly, and S. M. Wild, “Derivative -free optimization methods,” Acta Numerica, vol. 28, pp. 287–404, 2019, doi: DOI: 10.1017/S0962492919000060

work page doi:10.1017/s0962492919000060 2019

[13] [13]

A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation,

M. J. D. Powell, “A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation,” in Advances in Optimization and Numerical Analysis, S. Gomez and J.-P. Hennart, Eds., Dordrecht: Springer Netherlands, 1994, pp. 51–

work page 1994

[14] [14]

doi: 10.1007/978-94-015-8330-5_4

work page doi:10.1007/978-94-015-8330-5_4

[15] [15]

The BOBYQA algorithm for bound constrained optimization without derivatives,

M. J. D. Powell, “The BOBYQA algorithm for bound constrained optimization without derivatives,” Cambridge NA Report NA2009/06, University of Cambridge, Cambridge, vol. 26, pp. 26–46, 2009

work page 2009

[16] [16]

UOBYQA: unconstrained optimization by quadratic approximation,

M. J. D. Powell, “UOBYQA: unconstrained optimization by quadratic approximation,” Math Program, vol. 92, no. 3, pp. 555–582, 2002, doi: 10.1007/s101070100290

work page doi:10.1007/s101070100290 2002

[17] [17]

The NEWUOA software for unconstrained optimization without derivatives,

M. J. D. Powell, “The NEWUOA software for unconstrained optimization without derivatives,” in Large-Scale Nonlinear Optimization , G. Di Pillo and M. Roma, Eds., Boston, MA: Springer US, 2006, pp. 255–297. doi: 10.1007/0-387-30065-1_16

work page doi:10.1007/0-387-30065-1_16 2006

[18] [18]

Le Digabel, C

S. Le Digabel, C. Tribes, and C. Audet, NOMAD User Guide: Version 3.6. 0. GERAD HEC Montréal, 2013

work page 2013

[19] [19]

Dakota 6.22.0 documentation,

B. M. Adams et al. , “Dakota 6.22.0 documentation,” Albuquerque, NM, Nov. 2024. Accessed: Jul. 01, 2025. [Online]. Available: https://snl-dakota.github.io/

work page 2024

[20] [20]

Model-Based Derivative-Free Optimization Methods andSoftware,

T. M. Ragonneau, “Model-based derivative-free optimization methods and software,” arXiv preprint arXiv:2210.12018, 2022

work page arXiv 2022

[21] [21]

Derivative -free optimization: a review of algorithms and comparison of software implementations,

L. M. Rios and N. V Sahinidis, “Derivative -free optimization: a review of algorithms and comparison of software implementations,” Journal of Global Optimization, vol. 56, no. 3, pp. 1247–1293, 2013, doi: 10.1007/s10898-012-9951-y

work page doi:10.1007/s10898-012-9951-y 2013

[22] [22]

A Trust Region Filter Algorithm for Surrogate-based Optimization,

J. P. Eason, “A Trust Region Filter Algorithm for Surrogate-based Optimization,” Doctoral Dissertation, Carnegie Mellon University, Feb. 2018, doi: 10.1184/R1/6714431.v1

work page doi:10.1184/r1/6714431.v1 2018

[23] [23]

A. R. Conn, N. I. M. Gould, and P. L. Toint, Trust Region Methods. Society for Industrial and Applied Mathematics, 2000. doi: 10.1137/1.9780898719857

work page doi:10.1137/1.9780898719857 2000

[24] [24]

Nonlinear programming without a penalty function,

R. Fletcher and S. Leyffer, “Nonlinear programming without a penalty function,” Math Program, vol. 91, no. 2, pp. 239–269, 2002, doi: 10.1007/s101070100244. Page 32 of 40

work page doi:10.1007/s101070100244 2002

[25] [25]

A NOVEL FILTER TRUST-REGION ALGORITHM FOR CONSTRAINED OPTIMIZATION USING REDUCED ORDER MODELING,

A. Agarwal and L. T. Biegler, “A NOVEL FILTER TRUST-REGION ALGORITHM FOR CONSTRAINED OPTIMIZATION USING REDUCED ORDER MODELING,” Chemical Engineering Department, Carnegie Mellon University, Accessed: Jan. 29, 2024. [Online]. Available: http://focapo.cheme.cmu.edu/2012/proceedings/data/papers/040.pdf

work page 2024

[26] [26]

A trust-region framework for constrained optimization using reduced order modeling,

A. Agarwal and L. T. Biegler, “A trust-region framework for constrained optimization using reduced order modeling,” Optimization and Engineering , vol. 14, no. 1, pp. 3 –35, 2013, doi: 10.1007/s11081-011-9164-0

work page doi:10.1007/s11081-011-9164-0 2013

[27] [27]

On an inexact trust -region SQP-filter method for constrained nonlinear optimization,

A. Walther and L. Biegler, “On an inexact trust -region SQP-filter method for constrained nonlinear optimization,” Comput Optim Appl , vol. 63, no. 3, pp. 613 –638, 2016, doi: 10.1007/s10589-015-9793-x

work page doi:10.1007/s10589-015-9793-x 2016

[28] [28]

Demand‐based optimization of a chlorobenzene process with high‐fidelity and surrogate reactor models under trust region strategies,

N. Yoshio and L. T. Biegler, “Demand‐based optimization of a chlorobenzene process with high‐fidelity and surrogate reactor models under trust region strategies,” AIChE Journal, vol. 67, no. 1, p. e17054, 2021

work page 2021

[29] [29]

Optimization strategies for produced water networks with integrated desalination facilities,

S. Naik, M. Zamarripa, M. Drouven, and L. T. Biegler, “Optimization strategies for produced water networks with integrated desalination facilities,” Comput Chem Eng, vol. 187, p. 108738, 2024, doi: https://doi.org/10.1016/j.compchemeng.2024.108738

work page doi:10.1016/j.compchemeng.2024.108738 2024

[30] [30]

Efficient Trust Region Filter Modeling Strategies for Computationally Expensive Black -Box Optimization,

R. Liang, Y. Han, H. Hu, B. Chen, Z. Yuan, and L. T. Biegler, “Efficient Trust Region Filter Modeling Strategies for Computationally Expensive Black -Box Optimization,” Comput Chem Eng, p. 108816, 2024, doi: https://doi.org/10.1016/j.compchemeng.2024.108816

work page doi:10.1016/j.compchemeng.2024.108816 2024

[31] [31]

The trust region filter strategy: Survey of a rigorous approach for optimization with surrogate models,

L. T. Biegler, “The trust region filter strategy: Survey of a rigorous approach for optimization with surrogate models,” Digital Chemical Engineering , vol. 13, p. 100197, 2024, doi: https://doi.org/10.1016/j.dche.2024.100197

work page doi:10.1016/j.dche.2024.100197 2024

[32] [32]

Graduate Texts in Mathematics, vol 67

M. G. Ierapetritou and E. N. Pistikopoulos, “Global Optimization for Stochastic Planning, Scheduling and Design Problems,” in Global Optimization in Engineering Design , I. E. Grossmann, Ed., Boston, MA: Springer US, 1996, pp. 231–287. doi: 10.1007/978-1-4757- 5331-8_8

work page doi:10.1007/978-1-4757- 1996

[33] [33]

Hybrid simulation based optimization approach for supply chain management,

A. Nikolopoulou and M. G. Ierapetritou, “Hybrid simulation based optimization approach for supply chain management,” Comput Chem Eng , vol. 47, pp. 183 –193, 2012, doi: https://doi.org/10.1016/j.compchemeng.2012.06.045

work page doi:10.1016/j.compchemeng.2012.06.045 2012

[34] [34]

Chapter 5 - Data-driven optimization algorithms,

B. Beykal and E. N. Pistikopoulos, “Chapter 5 - Data-driven optimization algorithms,” in Artificial Intelligence in Manufacturing , M. Soroush and R. D Braatz, Eds., Academic Press, 2024, pp. 135–180. doi: https://doi.org/10.1016/B978-0-323-99134-6.00006-2

work page doi:10.1016/b978-0-323-99134-6.00006-2 2024

[35] [35]

Constrained optimization of black -box stochastic systems using a novel feasibility enhanced Kriging -based method,

Z. Wang and M. Ierapetritou, “Constrained optimization of black -box stochastic systems using a novel feasibility enhanced Kriging -based method,” Comput Chem Eng , vol. 118, pp. 210–223, 2018, doi: https://doi.org/10.1016/j.compchemeng.2018.07.016

work page doi:10.1016/j.compchemeng.2018.07.016 2018

[36] [36]

ARGONAUT: AlgoRithms for Global Optimization of coNstrAined grey-box compUTational problems,

F. Boukouvala and C. A. Floudas, “ARGONAUT: AlgoRithms for Global Optimization of coNstrAined grey-box compUTational problems,” Optim Lett, vol. 11, no. 5, pp. 895–913, 2017, doi: 10.1007/s11590-016-1028-2

work page doi:10.1007/s11590-016-1028-2 2017

[37] [37]

Global optimization of grey-box computational systems using surrogate functions and application Page 33 of 40 to highly constrained oil-field operations,

B. Beykal, F. Boukouvala, C. A. Floudas, N. Sorek, H. Zalavadia, and E. Gildin, “Global optimization of grey-box computational systems using surrogate functions and application Page 33 of 40 to highly constrained oil-field operations,” Comput Chem Eng, vol. 114, pp. 99–110, 2018, doi: https://doi.org/10.1016/j.compchemeng.2018.01.005

work page doi:10.1016/j.compchemeng.2018.01.005 2018

[38] [38]

Learning surrogate models for simulation - based optimization,

A. Cozad, N. V Sahinidis, and D. C. Miller, “Learning surrogate models for simulation - based optimization,” AIChE Journal , vol. 60, no. 6, pp. 2211 –2227, Jun. 2014, doi: https://doi.org/10.1002/aic.14418

work page doi:10.1002/aic.14418 2014

[39] [39]

The ALAMO approach to machine learning,

Z. T. Wilson and N. V Sahinidis, “The ALAMO approach to machine learning,” Comput Chem Eng , vol. 106, pp. 785 –795, 2017, doi: https://doi.org/10.1016/j.compchemeng.2017.02.010

work page doi:10.1016/j.compchemeng.2017.02.010 2017

[40] [40]

Deterministic global derivative -free optimization of black - box problems with bounded Hessian,

I. Bajaj and M. M. F. Hasan, “Deterministic global derivative -free optimization of black - box problems with bounded Hessian,” Optim Lett, vol. 14, no. 4, pp. 1011–1026, 2020, doi: 10.1007/s11590-019-01421-0

work page doi:10.1007/s11590-019-01421-0 2020

[41] [41]

A trust region -based two phase algorithm for constrained black-box and grey-box optimization with infeasible initial point,

I. Bajaj, S. S. Iyer, and M. M. Faruque Hasan, “A trust region -based two phase algorithm for constrained black-box and grey-box optimization with infeasible initial point,” Comput Chem Eng , vol. 116, pp. 306 –321, 2018, doi: https://doi.org/10.1016/j.compchemeng.2017.12.011

work page doi:10.1016/j.compchemeng.2017.12.011 2018

[42] [42]

ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions,

S. M. Wild, R. G. Regis, and C. A. Shoemaker, “ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions,” SIAM Journal on Scientific Computing , vol. 30, no. 6, pp. 3197–3219, 2008, doi: 10.1137/070691814

work page doi:10.1137/070691814 2008

[43] [43]

Trust regions in Kriging -based optimization with expected improvement,

R. G. Regis, “Trust regions in Kriging -based optimization with expected improvement,” Engineering Optimization , vol. 48, no. 6, pp. 1037 –1059, Jun. 2016, doi: 10.1080/0305215X.2015.1082350

work page doi:10.1080/0305215x.2015.1082350 2016

[44] [44]

Inexact Restoration method for nonlinear optimization without derivatives,

Ma. B. Arouxét, N. E. Echebest, and E. A. Pilotta, “Inexact Restoration method for nonlinear optimization without derivatives,” J Comput Appl Math , vol. 290, pp. 26 –43, 2015, doi: https://doi.org/10.1016/j.cam.2015.04.047

work page doi:10.1016/j.cam.2015.04.047 2015

[45] [45]

Advances in surrogate based modeling, feasibility analysis, and optimization: A review,

A. Bhosekar and M. Ierapetritou, “Advances in surrogate based modeling, feasibility analysis, and optimization: A review,” Comput Chem Eng , vol. 108, pp. 250 –267, 2018, doi: https://doi.org/10.1016/j.compchemeng.2017.09.017

work page doi:10.1016/j.compchemeng.2017.09.017 2018

[46] [46]

Neufang, E

M. Neufang, E. Pajak, D. van de Berg, Y. S. Lee, and E. A. del Rio Chanona, Surrogate- Based Optimization Techniques for Process Systems Engineering. December, 2024

work page 2024

[47] [47]

Surrogate model optimization: a comparison case study with pooling problems of CO2 point sources,

H. A. Pedrozo, M. A. Zamarripa, A. Uribe-Rodríguez, G. Panagakos, M. S. Diaz, and L. T. Biegler, “Surrogate model optimization: a comparison case study with pooling problems of CO2 point sources,” Comput Chem Eng , vol. 200, p. 109199, 2025, doi: https://doi.org/10.1016/j.compchemeng.2025.109199

work page doi:10.1016/j.compchemeng.2025.109199 2025

[48] [48]

Synthesis of Combined Heat and Mass Exchange Networks Via a Trust Region Filter Optimisation Algorithm Including Detailed Unit Designs,

S. R. Kazi, M. Short, and L. T. Biegler, “Synthesis of Combined Heat and Mass Exchange Networks Via a Trust Region Filter Optimisation Algorithm Including Detailed Unit Designs,” in Computer Aided Chemical Engineering, vol. 50, M. Türkay and R. Gani, Eds., Elsevier, 2021, pp. 13–18. doi: https://doi.org/10.1016/B978-0-323-88506-5.50003-6

work page doi:10.1016/b978-0-323-88506-5.50003-6 2021

[49] [49]

A trust -region framework for managing the use of approximation models in optimization,

N. M. Alexandrov, J. E. Dennis, R. M. Lewis, and V. Torczon, “A trust -region framework for managing the use of approximation models in optimization,” Structural optimization, vol. 15, no. 1, pp. 16–23, 1998, doi: 10.1007/BF01197433. Page 34 of 40

work page doi:10.1007/bf01197433 1998

[50] [50]

Benchmarking optimization software with COPS 3.0.,

E. D. Dolan, J. J. Moré, and T. S. Munson, “Benchmarking optimization software with COPS 3.0.,” Argonne National Lab., Argonne, IL (US), 2004

work page 2004

[51] [51]

Stabler Neo -Hookean Simulation: Absolute Eigenvalue Filtering for Projected Newton,

H. Chen, H.-T. D. Liu, D. I. W. Levin, C. Zheng, and A. Jacobson, “Stabler Neo -Hookean Simulation: Absolute Eigenvalue Filtering for Projected Newton,” in ACM SIGGRAPH 2024 Conference Papers , in SIGGRAPH ’24. New York, NY, USA: Association for Computing Machinery, 2024. doi: 10.1145/3641519.3657433

work page doi:10.1145/3641519.3657433 2024

[52] [52]

Trust -Region Eigenvalue Filtering for Projected Newton,

H. Chen, H. -T. D. Liu, A. Jacobson, D. I. W. Levin, and C. Zheng, “Trust -Region Eigenvalue Filtering for Projected Newton,” in SIGGRAPH Asia 2024 Conference Papers, in SA ’24. New York, NY, USA: Association for Computing Machinery, 2024. doi: 10.1145/3680528.3687650

work page doi:10.1145/3680528.3687650 2024

[53] [53]

Taking the human out of the loop: A review of Bayesian optimization,

B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. De Freitas, “Taking the human out of the loop: A review of Bayesian optimization,” Proceedings of the IEEE, vol. 104, no. 1, pp. 148–175, 2016, doi: 10.1109/JPROC.2015.2494218

work page doi:10.1109/jproc.2015.2494218 2016

[54] [54]

When Gaussian Process Meets Big Data: A Review of Scalable GPs,

H. Liu, Y.-S. Ong, X. Shen, and J. Cai, “When Gaussian Process Meets Big Data: A Review of Scalable GPs,” IEEE Trans Neural Netw Learn Syst , vol. 31, no. 11, pp. 4405 –4423, 2020, doi: 10.1109/TNNLS.2019.2957109

work page doi:10.1109/tnnls.2019.2957109 2020

[55] [55]

C. K. I. Williams and C. E. Rasmussen, Gaussian processes for machine learning , vol. 2, no. 3. MIT press Cambridge, MA, 2006

work page 2006

[56] [56]

Determination of eigenvectors with Lagrange multipliers,

W. Han, D. -W. Jung, J. Lee, and C. Yu, “Determination of eigenvectors with Lagrange multipliers,” Journal of the Korean Physical Society, vol. 78, no. 11, pp. 1018–1022, 2021, doi: 10.1007/s40042-021-00112-3

work page doi:10.1007/s40042-021-00112-3 2021

[57] [57]

Generating Eigenvalue Bounds Using Optimization,

H. Wolkowicz, “Generating Eigenvalue Bounds Using Optimization,” in Nonlinear Analysis and Variational Problems: In Honor of George Isac, P. M. Pardalos, T. M. Rassias, and A. A. Khan, Eds., New York, NY: Springer New York, 2010, pp. 465 –490. doi: 10.1007/978-1-4419-0158-3_29

work page doi:10.1007/978-1-4419-0158-3_29 2010

[58] [58]

Pyomo: modeling and solving mathematical programs in Python,

W. E. Hart, J.-P. Watson, and D. L. Woodruff, “Pyomo: modeling and solving mathematical programs in Python,” Math Program Comput , vol. 3, no. 3, pp. 219 –260, 2011, doi: 10.1007/s12532-011-0026-8

work page doi:10.1007/s12532-011-0026-8 2011

[59] [59]

A survey of the trust region subproblem within a semidefinite framework,

C. Fortin, “A survey of the trust region subproblem within a semidefinite framework,” University of Waterloo, Waterloo, Ontario, Canada, 2000. Accessed: May 15, 2025. [Online]. Available: http://hdl.handle.net/10012/1038

work page 2000

[60] [60]

A shifted Steihaug -Toint method for computing a trust-region step,

L. Lukšan, C. Matonoha, and J. Vlcek, “A shifted Steihaug -Toint method for computing a trust-region step,” Technická zpráva, vol. 914, 2004

work page 2004

[61] [61]

Convergence conditions for ascent methods,

P. Wolfe, “Convergence conditions for ascent methods,” SIAM review, vol. 11, no. 2, pp. 226–235, 1969

work page 1969

[62] [62]

Diagonally Perturbing a Symmetric Matrix to Make It Positive Definite,

N. Higham, “Diagonally Perturbing a Symmetric Matrix to Make It Positive Definite,” Research. Accessed: May 15, 2025. [Online]. Available: https://nhigham.com/2021/02/16/diagonally-perturbing-a-symmetric-matrix-to-make-it- positive-definite/?utm_source=chatgpt.com

work page 2025

[63] [63]

Trust -region algorithms for training responses: machine learning methods using indefinite Hessian approximations,

J. B. Erway, G. Joshua, M. Roummel F., and R. and Omheni, “Trust -region algorithms for training responses: machine learning methods using indefinite Hessian approximations,” Page 35 of 40 Optim Methods Softw , vol. 35, no. 3, pp. 460 –487, May 2020, doi: 10.1080/10556788.2019.1624747

work page doi:10.1080/10556788.2019.1624747 2020

[64] [64]

Robust quasistatic finite elements and flesh simulation,

J. Teran, E. Sifakis, G. Irving, and R. Fedkiw, “Robust quasistatic finite elements and flesh simulation,” in Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, 2005, pp. 181–190

work page 2005

[65] [65]

R. T. Rockafellar, Convex analysis, vol. 28. Princeton university press, 1997

work page 1997

[66] [66]

Numerical Optimization

J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed. New York, NY: Springer New York, NY, 2006. doi: https://doi.org/10.1007/978-0-387-40065-5

work page doi:10.1007/978-0-387-40065-5 2006

[67] [67]

gulhameed361/TRF -Solver: TRF -Solver v0.0.0,

G. Hameed, “gulhameed361/TRF -Solver: TRF -Solver v0.0.0,” Nov. 2025, Zenodo. doi: 10.5281/zenodo.17639336

work page doi:10.5281/zenodo.17639336 2025

[68] [68]

Virtual Library of Simulation Experiments: Test Functions and Datasets

S. Surjanovic and D. Bingham, “Virtual Library of Simulation Experiments: Test Functions and Datasets.” Accessed: Oct. 28, 2025. [Online]. Available: https://www.sfu.ca/~ssurjano/index.html

work page 2025

[69] [69]

COBALT: COnstrained Bayesian optimizAtion of computationaLly expensive grey-box models exploiting derivaTive information,

J. A. Paulson and C. Lu, “COBALT: COnstrained Bayesian optimizAtion of computationaLly expensive grey-box models exploiting derivaTive information,” Comput Chem Eng , vol. 160, p. 107700, 2022, doi: https://doi.org/10.1016/j.compchemeng.2022.107700

work page doi:10.1016/j.compchemeng.2022.107700 2022

[70] [70]

Yarin Gal, Riashat Islam, and Zoubin Ghahramani

E. D. Dolan and J. J. Moré, “Benchmarking optimization software with performance profiles,” Math Program, vol. 91, no. 2, pp. 201–213, 2002, doi: 10.1007/s101070100263

work page doi:10.1007/s101070100263 2002

[71] [71]

A penalty -based algorithm proposal for engineering optimization problems,

G. Z. Oztas and S. Erdem, “A penalty -based algorithm proposal for engineering optimization problems,” Neural Comput Appl , vol. 35, no. 10, pp. 7635 –7658, 2023, doi: 10.1007/s00521-022-08058-8

work page doi:10.1007/s00521-022-08058-8 2023

[72] [72]

T. F. Edgar, D. M. Himmelblau, and L. S. Lasdon, Optimization of Chemical Processes , 2nd ed. McGraw-Hill Chemical Engineering Series, 2001

work page 2001

[73] [73]

A Benchmark of PDF Information Extraction Tools Using a Multi-task and Multi- domain Evaluation Framework for Academic Documents,

A. H. Gandomi and X. -S. Yang, “Benchmark Problems in Structural Optimization,” in Computational Optimization, Methods and Algorithms , S. Koziel and X. -S. Yang, Eds., Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 259 –281. doi: 10.1007/978 -3- 642-20859-1_12

work page doi:10.1007/978 2011

[74] [74]

L. T. Biegler, Nonlinear programming: concepts, algorithms, and applications to chemical processes. Philadelphia: SIAM, 2010

work page 2010

[75] [75]

GitHub Repository: Pyomo,

N. Yoshio, J. Eason, and L. T. Biegler, “GitHub Repository: Pyomo,” Trust Region Framework Method Solver. Accessed: Oct. 27, 2025. [Online]. Available: https://pyomo.readthedocs.io/en/6.8.1/explanation/solvers/trustregion.html Appendix A: Global Convergence Eason and Biegler established that the sequence of iterates generated by the classical TRF algorith...

work page 2025

[76] [76]

If 𝜃(𝑘+1)≤ (1−𝛾𝜃)𝜃(𝑘) for all 𝑘, then 𝜃(𝑘)→ 0 for 𝑘→∞

work page

[77] [77]

Summing from 𝑘= 𝑘′ to 𝑘=𝑘′+𝑁 gives ∑ (𝑓(𝑘)−𝑓(𝑘+1)) 𝑘′+𝑁 𝑘=𝑘′ ≥ 𝛾𝑓 ∑ 𝜃(𝑘) 𝑘′+𝑁 𝑘=𝑘′

Otherwise, there exists an infinite subsequence 𝑘′ such that 𝑓(𝑘)−𝑓(𝑘+1)≥ 𝛾𝑓𝜃(𝑘) for all 𝑘≥ 𝑘′. Summing from 𝑘= 𝑘′ to 𝑘=𝑘′+𝑁 gives ∑ (𝑓(𝑘)−𝑓(𝑘+1)) 𝑘′+𝑁 𝑘=𝑘′ ≥ 𝛾𝑓 ∑ 𝜃(𝑘) 𝑘′+𝑁 𝑘=𝑘′ . Thus, 𝑓(𝑘′)−𝑓(𝑘′+𝑁+1) ≥ 𝛾𝑓 ∑ 𝜃(𝑘) 𝑘′+𝑁 𝑘=𝑘′ . Since 𝑓 is bounded below and monotonically decreasing, ∑ 𝜃(𝑘)𝑘′+𝑁 𝑘=𝑘′ < ∞ for 𝑁→ ∞, implying 𝜃(𝑘)→0 as 𝑘→∞. Therefore, 𝑙𝑖𝑚 𝑘→∞ 𝜃(...

work page 2000