Porting the Nonlinear Optimization Library HiOp to Accelerator-Based Hardware Architectures

Asher J. Mancinelli; Cosmin G. Petra; Jake Ryan; Kalyan S. Perumalla; Maksudul Alam; R. Cameron Rutherford; Slaven Peles

arxiv: 2605.13736 · v1 · pith:AL5QVHH3 · submitted 2026-05-13 · cs.MS

Porting the Nonlinear Optimization Library HiOp to Accelerator-Based Hardware Architectures

Slaven Peles , Kalyan S. Perumalla , Maksudul Alam , Asher J. Mancinelli , R. Cameron Rutherford , Jake Ryan , Cosmin G. Petra This is my paper

Reviewed by Pith2026-06-30 21:44 UTCgrok-4.3pith:AL5QVHH3 open to challenge →

classification cs.MS

keywords interior point methodsGPU accelerationnonlinear optimizationoptimal power flowsparse to dense compressionperformance portabilityaccelerator hardware

0 comments

The pith

A new formulation lets interior point methods run entirely on GPU accelerators by compressing sparse linear problems into small dense ones.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a version of the interior point method inside the HiOp library that executes completely on hardware accelerators such as GPUs. It does so by reformulating the algorithm to turn the sparse symmetric indefinite linear systems that normally arise into smaller dense problems, removing any dependence on sparse linear solvers. The work focuses on optimal power flow problems from power systems engineering and shows that the resulting dense problems stay small enough to fit accelerator memory while preserving convergence. It also examines the use of portability libraries Umpire and RAJA and the resulting trade-offs among speed, code portability, and developer effort. A reader would care because most modern computing power now sits in accelerators, yet interior point methods have been hard to move there until this compression step.

Core claim

We present a novel formulation of an interior point method implemented in our HiOp library, which is designed to be able to run entirely on hardware accelerators. This formulation avoids dependence on sparse solvers altogether, which is achieved by compressing the underlying sparse linear problem into a dense one of manageable size. We demonstrate feasibility of this approach and provide a baseline for future interior point method implementations on hardware accelerators. Our investigation is motivated by problems arising in optimal power flow analysis in power systems engineering and our approach is tailored to the broad class of problems arising in that important domain. We also demonstrat

What carries the argument

Compression of the sparse linear problem arising in each interior-point iteration into a dense problem whose size remains small enough for accelerator memory and compute.

If this is right

Interior point methods for sparse problems become usable on systems where more than 90 percent of processing power resides in GPUs.
The same compression approach applies to the broad class of optimal power flow problems in power systems engineering.
Performance-portability libraries such as Umpire and RAJA can be used to implement the method while trading off raw speed against development effort.
Future interior-point implementations on accelerators now have a concrete baseline that does not require sparse direct solvers.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The compression technique might extend to other engineering domains that produce similarly structured sparse KKT systems if their dense equivalents also remain modest in size.
The same porting strategy could reduce the need for specialized sparse solver libraries inside other nonlinear programming packages that target accelerators.
Hybrid CPU-GPU workflows might still be needed when the dense compression step itself becomes a bottleneck on very large instances.

Load-bearing premise

The sparse linear problems that appear in the target optimal power flow cases can be turned into dense problems small enough for GPU memory without destroying numerical stability or convergence of the interior point method.

What would settle it

Apply the compressed formulation to a representative set of optimal power flow test cases; if any iteration produces a dense matrix that exceeds available GPU memory or if the interior-point iteration count or final residual diverges from the CPU reference, the claim fails.

Figures

Figures reproduced from arXiv: 2605.13736 by Asher J. Mancinelli, Cosmin G. Petra, Jake Ryan, Kalyan S. Perumalla, Maksudul Alam, R. Cameron Rutherford, Slaven Peles.

**Figure 2.** Figure 2: Performance before and after parallelization with RAJA [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗

**Figure 3.** Figure 3: The line in the plot represents the base roofline collected from running micro kernels on the device. The closer a point is to the line, the more optimized it is with respect to performance that can be achieved for the given arithmetic intensity. Based on this observation, it can be seen that most of the routines are fairly optimized, [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

read the original abstract

While interior point methods have been the centerpiece of nonlinear programming tools used in science and engineering, their reliance on linear solvers that can tackle sparse symmetric indefinite and highly ill-conditioned problems made it difficult to implement them effectively on hardware accelerators. At this time, there are few sparse linear solvers that can be used in this context. Here, we present a novel formulation of an interior point method implemented in our HiOp library, which is designed to be able to run entirely on hardware accelerators. This formulation avoids dependence on sparse solvers altogether, which is achieved by compressing the underlying sparse linear problem into a dense one of manageable size. We demonstrate feasibility of this approach and provide a baseline for future interior point method implementations on hardware accelerators. Our investigation is motivated by problems arising in optimal power flow analysis in power systems engineering and our approach is tailored to the broad class of problems arising in that important domain. We also demonstrate utility of modern programming models based on performance portability libraries, namely, Umpire and RAJA. We discuss trade-offs between performance, portability and development cost in the solution space for this non-linear optimization problem. As a result of this research, we demonstrate for the first time that interior point methods for sparse problems can be efficiently realized on modern computing systems where more than 90% of processing power is in GPUs.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper's compression of sparse KKT systems into dense form is a concrete step toward GPU-only IPMs, but the abstract supplies no sizes, timings, or convergence data to show the trick actually works.

read the letter

The main thing here is a reformulation that compresses the sparse symmetric indefinite KKT systems from interior point methods into dense matrices small enough to run entirely on accelerators. This removes the dependence on sparse solvers that are hard to find or inefficient on GPUs, and the authors target it at optimal power flow problems that can tolerate the step.

They handle the implementation with Umpire and RAJA for portability, and they lay out the trade-offs between performance, portability, and coding effort in a way that feels grounded in actual development experience. That part is useful for anyone facing similar hardware shifts.

The soft spot is the missing evidence. The abstract states a working demonstration and a first-time result, yet it gives no matrix dimensions after compression, no iteration counts, no residual histories, and no direct comparison to a sparse reference solver. Without those, it is impossible to check whether the dense problems stay manageable in memory, preserve numerical stability, or keep the same convergence behavior on the target instances. The stress-test note correctly flags that the central claim rests on an untested assumption about the compression outcome.

This paper is aimed at researchers porting nonlinear optimization codes to GPU-heavy systems, especially in power systems and other sparse NLP domains. A reader already working on accelerator ports would pick up the formulation idea and the portability discussion. It deserves a serious referee because the technique addresses a documented practical barrier with a specific mechanism, even though the experimental section will likely need substantial expansion to support the efficiency claim.

Referee Report

2 major / 0 minor

Summary. The manuscript presents a novel formulation of an interior point method (IPM) in the HiOp library that avoids sparse linear solvers by compressing the underlying sparse KKT systems into dense problems of manageable size. This enables the solver to run entirely on accelerator hardware such as GPUs. The work is motivated by optimal power flow (OPF) instances in power systems engineering, uses performance-portability libraries Umpire and RAJA, discusses trade-offs among performance, portability, and development cost, and claims to demonstrate for the first time that IPMs for sparse problems can be efficiently realized on modern systems where >90% of processing power resides in GPUs.

Significance. If the compression operator produces dense matrices that remain small enough for GPU memory, preserve numerical stability and convergence behavior of the IPM, and still solve the original OPF instances to required accuracy, the result would supply a concrete baseline for accelerator-native nonlinear optimization and open a path for deploying IPMs on GPU-heavy architectures. The emphasis on portability libraries and explicit discussion of development-cost trade-offs would also be useful to the mathematical-software community.

major comments (2)

Abstract: The central claim that the approach demonstrates 'for the first time' that IPMs for sparse problems can be efficiently realized on GPU-dominated systems rests on the unverified assumption that sparse-to-dense compression yields GPU-fit matrices while preserving IPM convergence and stability. No matrix dimensions after compression, iteration counts, residual histories, timing tables, or comparison against a sparse reference solver are supplied, leaving the weakest assumption untested even at the level of reported outcomes.
Formulation and results sections: The description of the compression operator itself is absent. Without an explicit statement of how the sparse KKT system is mapped to a dense one (including any fill-in control or numerical safeguards), it is impossible to assess whether the method is general for the claimed broad class of OPF problems or merely ad-hoc for the instances considered.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed review. The comments highlight areas where additional explicit detail will strengthen the manuscript, and we will revise accordingly.

read point-by-point responses

Referee: Abstract: The central claim that the approach demonstrates 'for the first time' that IPMs for sparse problems can be efficiently realized on GPU-dominated systems rests on the unverified assumption that sparse-to-dense compression yields GPU-fit matrices while preserving IPM convergence and stability. No matrix dimensions after compression, iteration counts, residual histories, timing tables, or comparison against a sparse reference solver are supplied, leaving the weakest assumption untested even at the level of reported outcomes.

Authors: We agree that the submitted manuscript does not supply the requested quantitative details (post-compression dimensions, iteration counts, residual histories, timing tables, or sparse-solver comparisons). While the work demonstrates successful end-to-end GPU execution on representative OPF instances, these supporting metrics were not reported. We will revise the abstract to include key metrics and add a results subsection with the missing data, including compressed matrix sizes that fit GPU memory, convergence histories, and direct timing comparisons against a sparse reference solver. This will furnish the verification the referee correctly identifies as absent. revision: yes
Referee: Formulation and results sections: The description of the compression operator itself is absent. Without an explicit statement of how the sparse KKT system is mapped to a dense one (including any fill-in control or numerical safeguards), it is impossible to assess whether the method is general for the claimed broad class of OPF problems or merely ad-hoc for the instances considered.

Authors: The referee correctly observes that the current text provides only a high-level description of the compression step. We will revise the formulation section to supply an explicit algorithmic statement of the operator: the precise mapping from the sparse KKT matrix to the dense reduced system, the use of the power-network incidence structure to limit fill-in, and the numerical safeguards (regularization of the diagonal blocks and optional iterative refinement) employed to preserve the convergence and stability properties of the original IPM. These additions will allow readers to evaluate generality for the broader OPF class. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical demonstration without self-referential derivations

full rationale

The paper's central contribution is an empirical demonstration of a compression-based reformulation of interior-point methods to enable GPU execution, motivated by optimal power flow problems. No equations, fitted parameters, or first-principles derivations are presented that reduce to their own inputs by construction. The approach relies on implementation details and performance measurements rather than any self-definitional or self-citation load-bearing steps. Self-references to the HiOp library are contextual and not used to justify uniqueness or force the result.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain-specific assumption that optimal power flow problems possess sufficient structure for the sparse-to-dense compression to remain both accurate and computationally tractable; no free parameters or invented entities are mentioned in the abstract.

axioms (1)

domain assumption Sparse linear systems arising in optimal power flow admit compression to dense matrices of manageable size without destroying the convergence properties of the interior point method.
This premise is required for the reformulation to eliminate sparse solvers while still solving the original problems.

pith-pipeline@v0.9.1-grok · 5791 in / 1284 out tokens · 28719 ms · 2026-06-30T21:44:06.469338+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages

[1]

´Swirydowicz, E

K. ´Swirydowicz, E. Darve, W. Jones, J. Maack, S. Regev, M. A. Saunders, S. J. Thomas, S. Peleš, Linear solvers for power grid optimization prob- lems: a review of gpu-accelerated linear solvers, Parallel Computing 111 (2022) 102870

work page 2022
[2]

L. T. Biegler, A survey on sensitivity-based nonlin- ear model predictive control, IFAC Proceedings V ol- umes 46 (32) (2013) 499–510

work page 2013
[3]

C. G. Petra, A memory-distributed quasi-newton solver for nonlinear programming problems with a small number of general constraints, Journal of Par. and Dist. Computing 133 (2019) 337–348

work page 2019
[4]

Z. Duan, M. Andronescu, K. Schutz, S. McIlwain, Y . J. Kim, C. Lee, J. Shendure, S. Fields, C. A. Blau, W. S. Noble, A three-dimensional model of the yeast genome, Nature 465 (7296) (2010) 363–367

work page 2010
[5]

X. Kai, Y . Song, Interior point methods and appli- cations in power systems, in: Modern Optimisation Techniques in Power Systems, Springer, 1999, pp. 205–245

work page 1999
[6]

Wächter, L

A. Wächter, L. T. Biegler, On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming, Mathematical programming 106 (1) (2006) 25–57

work page 2006
[7]

Chakrabarti, M

S. Chakrabarti, M. Kraning, E. Chu, R. Baldick, S. Boyd, Security constrained optimal power flow via proximal message passing, in: Clemson Univ. Power Systems Conference, IEEE, 2014, pp. 1–8. 13

work page 2014
[8]

C. G. Petra, O. Schenk, M. Anitescu, Real-time stochastic optimization of complex energy systems on high-performance computers, Computing in Sci- ence & Engineering 16 (5) (2014) 32–42

work page 2014
[9]

J. D. Booth, S. Rajamanickam, H. Thornquist, Basker: A threaded sparse lu factorization utilizing hierarchical parallelism and data layouts, in: 2016 IEEE International Parallel and Distributed Process- ing Symposium Workshops (IPDPSW), IEEE, 2016, pp. 673–682

work page 2016
[10]

Dorfler, F

F. Dorfler, F. Bullo, Kron reduction of graphs with applications to electrical networks, IEEE Transac- tions on Circuits and Systems I: Regular Papers 60 (1) (2013) 150–163

work page 2013
[11]

Wächter, L

A. Wächter, L. T. Biegler, Line search filter methods for nonlinear programming: Motivation and global convergence, SIAM Journal on Optimization 16 (1) (2005) 1–31

work page 2005
[12]

Wächter, L

A. Wächter, L. T. Biegler, Line search filter meth- ods for nonlinear programming: Local convergence, SIAM Journal on Optimization 16 (1) (2005) 32–48

work page 2005
[13]

Fletcher, S

R. Fletcher, S. Leyffer, Nonlinear programming without a penalty function, Mathematical Program- ming 91 (2) (2002) 239–269

work page 2002
[14]

J. R. Bunch, L. Kaufman, Some stable methods for calculating inertia and solving symmetric linear sys- tems, Mathematics of Computation 31 (137) (1977) 163–179

work page 1977
[15]

I. S. Duff, J. K. Reid, The multifrontal solution of in- definite sparse symmetric linear, ACM Trans. Math. Softw. 9 (3) (1983) 302–325

work page 1983
[16]

E. V . Haynsworth, Determination of the inertia of a partitioned Hermitian matrix, Linear Algebra and its Applications 1 (1) (1968) 73–81

work page 1968
[17]

Abhyankar, S

S. Abhyankar, S. Peles, R. Rutherford, A. Mancinelli, Evaluation of ac optimal power flow on graphical processing units, in: 2021 IEEE Power & Energy Society General Meeting (PESGM), IEEE, 2021, pp. 01–05

work page 2021
[18]

Tomov, J

S. Tomov, J. Dongarra, M. Baboulin, Towards dense linear algebra for hybrid GPU accelerated manycore systems, Parallel Computing 36 (5-6) (2010) 232– 240

work page 2010
[19]

M. A. Heroux, R. A. Bartlett, V . E. Howle, R. J. Hoekstra, J. J. Hu, T. G. Kolda, R. B. Lehoucq, K. R. Long, R. P. Pawlowski, E. T. Phipps, A. G. Salinger, H. K. Thornquist, R. S. Tuminaro, J. M. Willen- bring, A. Williams, K. S. Stanley, An overview of the trilinos project, ACM Trans. Math. Softw. 31 (3) (2005) 397–423

work page 2005
[20]

A. C. Hindmarsh, P. N. Brown, K. E. Grant, S. L. Lee, R. Serban, D. E. Shumaker, C. S. Wood- ward, SUNDIALS: Suite of nonlinear and differen- tial/algebraic equation solvers, ACM Transactions on Mathematical Software 31 (3) (2005) 363–396

work page 2005
[21]

S. B. at al., PETSc users manual, Tech. Rep. ANL- 95/11-Rev. 3.15, Argonne National Lab. (2021)

work page 2021
[22]

D. A. Beckingsale, M. J. Mcfadden, J. P. Dahm, R. Pankajakshan, R. D. Hornung, Umpire: Application-focused management and coordination of complex hierarchical memory, IBM Journal of Research and Development 64 (3/4) (2019) 00–1

work page 2019
[23]

D. A. Beckingsale, J. Burmark, R. Hornung, H. Jones, W. Killian, A. J. Kunen, O. Pearce, P. Robinson, B. S. Ryujin, T. R. Scogland, Raja: Portable performance for large-scale scientific ap- plications, in: 2019 IEEE/ACM International Work- shop on Performance, Portability and Productivity in HPC (P3HPC), IEEE, 2019, pp. 71–81

work page 2019
[24]

H. C. Edwards, C. R. Trott, D. Sunderland, Kokkos: Enabling manycore performance portability through polymorphic memory access patterns, Journal of Parallel and Distributed Computing 74 (12) (2014) 3202 – 3216

work page 2014
[25]

0 (2018)

NVIDIA, Cuda toolkit documentation v10. 0 (2018). 14

work page 2018

[1] [1]

´Swirydowicz, E

K. ´Swirydowicz, E. Darve, W. Jones, J. Maack, S. Regev, M. A. Saunders, S. J. Thomas, S. Peleš, Linear solvers for power grid optimization prob- lems: a review of gpu-accelerated linear solvers, Parallel Computing 111 (2022) 102870

work page 2022

[2] [2]

L. T. Biegler, A survey on sensitivity-based nonlin- ear model predictive control, IFAC Proceedings V ol- umes 46 (32) (2013) 499–510

work page 2013

[3] [3]

C. G. Petra, A memory-distributed quasi-newton solver for nonlinear programming problems with a small number of general constraints, Journal of Par. and Dist. Computing 133 (2019) 337–348

work page 2019

[4] [4]

Z. Duan, M. Andronescu, K. Schutz, S. McIlwain, Y . J. Kim, C. Lee, J. Shendure, S. Fields, C. A. Blau, W. S. Noble, A three-dimensional model of the yeast genome, Nature 465 (7296) (2010) 363–367

work page 2010

[5] [5]

X. Kai, Y . Song, Interior point methods and appli- cations in power systems, in: Modern Optimisation Techniques in Power Systems, Springer, 1999, pp. 205–245

work page 1999

[6] [6]

Wächter, L

A. Wächter, L. T. Biegler, On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming, Mathematical programming 106 (1) (2006) 25–57

work page 2006

[7] [7]

Chakrabarti, M

S. Chakrabarti, M. Kraning, E. Chu, R. Baldick, S. Boyd, Security constrained optimal power flow via proximal message passing, in: Clemson Univ. Power Systems Conference, IEEE, 2014, pp. 1–8. 13

work page 2014

[8] [8]

C. G. Petra, O. Schenk, M. Anitescu, Real-time stochastic optimization of complex energy systems on high-performance computers, Computing in Sci- ence & Engineering 16 (5) (2014) 32–42

work page 2014

[9] [9]

J. D. Booth, S. Rajamanickam, H. Thornquist, Basker: A threaded sparse lu factorization utilizing hierarchical parallelism and data layouts, in: 2016 IEEE International Parallel and Distributed Process- ing Symposium Workshops (IPDPSW), IEEE, 2016, pp. 673–682

work page 2016

[10] [10]

Dorfler, F

F. Dorfler, F. Bullo, Kron reduction of graphs with applications to electrical networks, IEEE Transac- tions on Circuits and Systems I: Regular Papers 60 (1) (2013) 150–163

work page 2013

[11] [11]

Wächter, L

A. Wächter, L. T. Biegler, Line search filter methods for nonlinear programming: Motivation and global convergence, SIAM Journal on Optimization 16 (1) (2005) 1–31

work page 2005

[12] [12]

Wächter, L

A. Wächter, L. T. Biegler, Line search filter meth- ods for nonlinear programming: Local convergence, SIAM Journal on Optimization 16 (1) (2005) 32–48

work page 2005

[13] [13]

Fletcher, S

R. Fletcher, S. Leyffer, Nonlinear programming without a penalty function, Mathematical Program- ming 91 (2) (2002) 239–269

work page 2002

[14] [14]

J. R. Bunch, L. Kaufman, Some stable methods for calculating inertia and solving symmetric linear sys- tems, Mathematics of Computation 31 (137) (1977) 163–179

work page 1977

[15] [15]

I. S. Duff, J. K. Reid, The multifrontal solution of in- definite sparse symmetric linear, ACM Trans. Math. Softw. 9 (3) (1983) 302–325

work page 1983

[16] [16]

E. V . Haynsworth, Determination of the inertia of a partitioned Hermitian matrix, Linear Algebra and its Applications 1 (1) (1968) 73–81

work page 1968

[17] [17]

Abhyankar, S

S. Abhyankar, S. Peles, R. Rutherford, A. Mancinelli, Evaluation of ac optimal power flow on graphical processing units, in: 2021 IEEE Power & Energy Society General Meeting (PESGM), IEEE, 2021, pp. 01–05

work page 2021

[18] [18]

Tomov, J

S. Tomov, J. Dongarra, M. Baboulin, Towards dense linear algebra for hybrid GPU accelerated manycore systems, Parallel Computing 36 (5-6) (2010) 232– 240

work page 2010

[19] [19]

M. A. Heroux, R. A. Bartlett, V . E. Howle, R. J. Hoekstra, J. J. Hu, T. G. Kolda, R. B. Lehoucq, K. R. Long, R. P. Pawlowski, E. T. Phipps, A. G. Salinger, H. K. Thornquist, R. S. Tuminaro, J. M. Willen- bring, A. Williams, K. S. Stanley, An overview of the trilinos project, ACM Trans. Math. Softw. 31 (3) (2005) 397–423

work page 2005

[20] [20]

A. C. Hindmarsh, P. N. Brown, K. E. Grant, S. L. Lee, R. Serban, D. E. Shumaker, C. S. Wood- ward, SUNDIALS: Suite of nonlinear and differen- tial/algebraic equation solvers, ACM Transactions on Mathematical Software 31 (3) (2005) 363–396

work page 2005

[21] [21]

S. B. at al., PETSc users manual, Tech. Rep. ANL- 95/11-Rev. 3.15, Argonne National Lab. (2021)

work page 2021

[22] [22]

D. A. Beckingsale, M. J. Mcfadden, J. P. Dahm, R. Pankajakshan, R. D. Hornung, Umpire: Application-focused management and coordination of complex hierarchical memory, IBM Journal of Research and Development 64 (3/4) (2019) 00–1

work page 2019

[23] [23]

D. A. Beckingsale, J. Burmark, R. Hornung, H. Jones, W. Killian, A. J. Kunen, O. Pearce, P. Robinson, B. S. Ryujin, T. R. Scogland, Raja: Portable performance for large-scale scientific ap- plications, in: 2019 IEEE/ACM International Work- shop on Performance, Portability and Productivity in HPC (P3HPC), IEEE, 2019, pp. 71–81

work page 2019

[24] [24]

H. C. Edwards, C. R. Trott, D. Sunderland, Kokkos: Enabling manycore performance portability through polymorphic memory access patterns, Journal of Parallel and Distributed Computing 74 (12) (2014) 3202 – 3216

work page 2014

[25] [25]

0 (2018)

NVIDIA, Cuda toolkit documentation v10. 0 (2018). 14

work page 2018