The reviewed record of science sign in
Pith

arxiv: 2606.13255 · v1 · pith:22CZE6V2 · submitted 2026-06-11 · cs.CE · cs.NA· math.NA

Embedding-based Methods for Linear Solver Performance Prediction

Hayden Liu Weng , Hans-Joachim Bungartz , Felix Dietrich This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-27 05:11 UTCgrok-4.3pith:22CZE6V2record.jsonopen to challenge →

classification cs.CE cs.NAmath.NA
keywords embedding-based solver selectionlinear solver performance predictionmultilabel predictionsparse matrix featuresSuiteSparse collectionPETSc configurationsMAPE and nDCG metrics
0
0 comments X

The pith

An embedding-based framework learns solver-problem relationships from performance data to predict optimal linear solvers more accurately than classical feature-based models.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes a modular framework that separates the learning of solver performance from the choice of matrix features, allowing inexpensive numerical features to map new problems into an embedding space built from observed solver runs. This separation matters because libraries offer over a hundred solver and preconditioner combinations whose relative speed varies sharply across sparse linear systems, yet picking the right one often requires expensive matrix analysis or exhaustive testing. Experiments across 621 SuiteSparse matrices and 101 PETSc configurations show the embedding approach raises top-prediction accuracy by 17 percent and cuts mean average percentage error by 37 percent when expensive features are available, while staying competitive and still 24 percent better on error metrics when restricted to cheap features only.

Core claim

The framework learns solver-problem relationships directly from observed performance data inside a shared embedding space; inexpensive numerical features then project unseen matrices into that same space so that a downstream multilabel predictor can rank the 101 solver configurations by expected performance, measured with user-centric metrics such as MAPE and nDCG rather than classification accuracy alone.

What carries the argument

The modular embedding space that decouples performance modeling from feature representation and downstream prediction.

If this is right

  • Solver selection can be performed without recomputing expensive matrix properties for every new problem.
  • The same embedding can support multilabel ranking that reflects relative runtime rather than binary success or failure.
  • Performance remains competitive when the feature budget is deliberately reduced.
  • The approach scales to 101 distinct PETSc configurations across hundreds of matrices from a standard public collection.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The learned embedding space may expose natural clusters of matrices that share similar solver preferences, enabling transfer to problems outside the original training distribution.
  • The decoupling of modeling and features could be applied to other configuration-selection tasks such as choosing preconditioners for iterative methods or selecting time-stepping schemes.
  • If the embedding is updated online with new performance measurements, the predictor could adapt to hardware changes or evolving problem distributions without retraining from scratch.

Load-bearing premise

Inexpensive numerical features suffice to place new matrices accurately inside an embedding space that was built only from performance observations on a training set of matrices.

What would settle it

Running the trained model on a fresh collection of matrices never seen during embedding construction or feature projection and finding that top-prediction accuracy and MAPE no longer improve over a classical feature-based baseline.

Figures

Figures reproduced from arXiv: 2606.13255 by Felix Dietrich, Hans-Joachim Bungartz, Hayden Liu Weng.

Figure 1
Figure 1. Figure 1: Proposed pipeline, where F represents the collected values of selected features, G the embeddings for the training samples, and α the coefficients for the projection operator. Offline training phase indicated by the marked region. have further expanded the portfolio of applicable techniques. While promis￾ing, these approaches introduce substantial training and per-instance inference overheads and may strug… view at source ↗
Figure 2
Figure 2. Figure 2: Number of problems successfully converging (left [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: PCA projections of the trained embeddings, colore [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Sorted relative performance of the reduced embedd [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗
read the original abstract

The solution of large, sparse linear systems often dominates the computational effort of scientific applications and is a frequent optimization target. Modern libraries provide numerous solver and preconditioner configurations, but their performance varies significantly across problem instances. Previous works have addressed the selection of an optimal solver, but are typically limited by the problem set addressed (e.g., only symmetric positive definite matrices), the use of expensive matrix features, or the complexity of the approach. This work proposes a modular, low-cost embedding-based framework for solver selection that decouples performance modeling from feature representation and downstream prediction. Solver-problem relationships are learned directly from observed performance data, while inexpensive numerical features are used to project unseen problems into the learned embedding space. The framework focuses on multilabel prediction and evaluation using user-centric metrics, such as MAPE and nDCG, which better reflect relative performance. Experiments on 621 matrices from the SuiteSparse matrix collection across 101 PETSc solver configurations demonstrate a 17% increase in top-prediction accuracy over classical feature-based models when expensive numerical features are included, along with reductions of 37% in mean average percentage error (MAPE) and 46% in top-prediction error (1-error). When restricted to a reduced feature set, the embedding approach remains competitive, while still consistently achieving ca. 24% lower MAPE and 1-error across a broad range of problems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes a modular embedding-based framework for linear solver performance prediction that decouples learning of solver-problem relationships from inexpensive numerical feature projection into the embedding space. It targets multilabel prediction using user-centric metrics (MAPE, nDCG, 1-error) and reports results on 621 SuiteSparse matrices across 101 PETSc configurations, claiming a 17% gain in top-prediction accuracy over classical feature-based models (with further gains when expensive features are added), plus 37% MAPE reduction and 46% 1-error reduction; the embedding approach remains competitive under reduced features.

Significance. If the central claims hold after addressing validation gaps, the work could advance practical solver selection in scientific computing by offering a flexible, low-cost alternative to expensive-feature or problem-restricted prior methods. The data-driven embedding from observed performance data and focus on multilabel/user-centric metrics are strengths; the modular decoupling, if isolated, would be a notable contribution over monolithic feature-based baselines.

major comments (2)
  1. [Section 3] Section 3: The framework overview claims decoupling of embedding learning from feature projection, yet no ablation is reported that holds the learned embedding fixed while varying only the inexpensive projection features (or measures embedding-space distortion for out-of-distribution matrices). Without this, the reported competitiveness under reduced features cannot be attributed specifically to the embedding approach rather than classical feature effects.
  2. [Section 4] Section 4 (experiments): Aggregate gains (17% top-accuracy, 37% MAPE drop) are presented across 621 matrices and 101 configurations, but the support for the projection assumption lacks isolated validation (e.g., no hold-out embedding test or distortion metric). This is load-bearing for the central claim that inexpensive features suffice to project unseen problems accurately.
minor comments (1)
  1. [Abstract] Abstract and Section 4: Clarify whether the 17% accuracy gain is measured with or without expensive features, and provide per-configuration breakdowns or variance to support the broad-range claim.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments. We address each major comment below and will revise the manuscript to include the requested isolated validations of the embedding decoupling and projection assumption.

read point-by-point responses

  1. Referee: [Section 3] Section 3: The framework overview claims decoupling of embedding learning from feature projection, yet no ablation is reported that holds the learned embedding fixed while varying only the inexpensive projection features (or measures embedding-space distortion for out-of-distribution matrices). Without this, the reported competitiveness under reduced features cannot be attributed specifically to the embedding approach rather than classical feature effects.

    Authors: We agree that the current manuscript lacks an explicit ablation that holds the learned embedding fixed while varying only the inexpensive projection features, and does not report embedding-space distortion metrics for out-of-distribution matrices. This limits the strength of the attribution to the embedding approach. In the revision we will add such an ablation study together with distortion analysis. revision: yes

  2. Referee: [Section 4] Section 4 (experiments): Aggregate gains (17% top-accuracy, 37% MAPE drop) are presented across 621 matrices and 101 configurations, but the support for the projection assumption lacks isolated validation (e.g., no hold-out embedding test or distortion metric). This is load-bearing for the central claim that inexpensive features suffice to project unseen problems accurately.

    Authors: We concur that the experiments section does not provide isolated validation such as a hold-out embedding test or distortion metric to directly support the projection assumption. This is a valid concern for the central claim. We will add these analyses in the revised manuscript. revision: yes

Circularity Check

0 steps flagged

No circularity detected in derivation or predictions

full rationale

The paper describes a modular embedding framework that learns solver-problem relationships directly from observed performance data on 621 SuiteSparse matrices across 101 PETSc configurations, then uses inexpensive numerical features only for projection into that space. No equations, fitted parameters, or claims reduce the reported multilabel predictions (MAPE, nDCG, top-accuracy) to inputs by construction. No self-citation chains, uniqueness theorems, or ansatzes are invoked as load-bearing; the approach is externally validated on held-out matrices with aggregate metrics. This is a standard data-driven ML setup with no reduction to self-definition or renaming of known results.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

Based on abstract only; the embedding space is a learned construct from performance data. Details on model architecture and training are absent, so free parameters and axioms cannot be fully enumerated.

free parameters (1)
  • embedding dimension and other ML hyperparameters
    Standard in embedding models but unspecified in abstract; likely fitted during training on performance data.
axioms (1)
  • domain assumption Performance observations on SuiteSparse matrices generalize to project new problems accurately via inexpensive features
    Core to the projection step in the framework.

pith-pipeline@v0.9.1-grok · 5784 in / 1248 out tokens · 40022 ms · 2026-06-27T05:11:52.485594+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 1 canonical work pages

  1. [1]

    Adams, Steven Benson, Jed Brown, Peter Brune, Kris Buschelman, Emil M

    Satish Balay, Shrirang Abhyankar, Mark F. Adams, Steven Benson, Jed Brown, Peter Brune, Kris Buschelman, Emil M. Constantinesc u, Lisan- dro Dalcin, Alp Dener, Victor Eijkhout, Jacob Faibussowits ch, William D. Gropp, Václav Hapla, Tobin Isaac, Pierre Jolivet, Dmitry Ka rpeev, Di- nesh Kaushik, Matthew G. Knepley, Fande Kong, Scott Kruger, Dave A. May, Lo...

    2024

  2. [2]

    Templates for the solution of linear systems: building blocks for iterative methods

    Richard Barrett, Michael Berry, Tony F Chan, James Demme l, June Do- nato, Jack Dongarra, Victor Eijkhout, Roldan Pozo, Charles Romine, and Henk Van der Vorst. Templates for the solution of linear systems: building blocks for iterative methods . SIAM, 1994

    1994

  3. [3]

    Preconditioning techniques for large li near systems: a sur- vey

    Michele Benzi. Preconditioning techniques for large li near systems: a sur- vey. Journal of computational Physics , 182(2):418–477, 2002

    2002

  4. [4]

    Application of machine learning to the selecti on of sparse linear solvers

    Sanjukta Bhowmick, Victor Eijkhout, Yoav Freund, Erika Fuentes, and David Keyes. Application of machine learning to the selecti on of sparse linear solvers. Int. J. High Perf. Comput. Appl , 2006

    2006

  5. [5]

    A n ew pair of gloves

    Riley Carlson, John Bauer, and Christopher D Manning. A n ew pair of gloves. arXiv preprint arXiv:2507.18103 , 2025

    work page arXiv 2025

  6. [6]

    Self- adapting software for numerical linear algebra and lapack f or clusters

    Zizhong Chen, Jack Dongarra, Piotr Luszczek, and Kennet h Roche. Self- adapting software for numerical linear algebra and lapack f or clusters. Par- allel Computing , 29(11-12):1723–1743, 2003. 14

    2003

  7. [7]

    The university of florida spa rse matrix collection

    Timothy A Davis and Yifan Hu. The university of florida spa rse matrix collection. ACM Transactions on Mathematical Software (TOMS) , 38(1):1– 25, 2011

    2011

  8. [8]

    Self-adapting numerical sof tware (sans) effort

    Jack Dongarra, George Bosilca, Zizhong Chen, Victor Eij khout, Gra- ham E Fagg, Erika Fuentes, Julien Langou, Piotr Luszczek, Je lena Pjesivac- Grbovic, Keith Seymour, et al. Self-adapting numerical sof tware (sans) effort. IBM Journal of Research and Development , 50(2.3):223–238, 2006

    2006

  9. [9]

    Self-adapting numer ical software for next generation applications

    Jack Dongarra and Victor Eijkhout. Self-adapting numer ical software for next generation applications. The International Journal of High Perfor- mance Computing Applications , 17(2):125–131, 2003

    2003

  10. [10]

    Preconditioning for sparse l inear systems at the dawn of the 21st century: History, current developments, an d future per- spectives

    Massimiliano Ferronato. Preconditioning for sparse l inear systems at the dawn of the 21st century: History, current developments, an d future per- spectives. International Scholarly Research Notices , 2012(1):127647, 2012

    2012

  11. [11]

    Predictio n of optimal solvers for sparse linear systems using deep learning

    Yannick Funk, Markus Götz, and Hartwig Anzt. Predictio n of optimal solvers for sparse linear systems using deep learning. In Proceedings of the 2022 SIAM Conference on Parallel Processing for Scientific Co mputing, pages 14–24. Society for Industrial and Applied Mathematic s, 2022

    2022

  12. [12]

    A recomme ndation sys- tem for preconditioned iterative solvers

    Thomas George, Anshul Gupta, and Vivek Sarin. A recomme ndation sys- tem for preconditioned iterative solvers. In 2008 Eighth IEEE International Conference on Data Mining , pages 803–808. IEEE, 2008

    2008

  13. [13]

    https://llnl.gov/casc/hypre, https://github.com/hypre-space/hypre

    hypre: High performance preconditioners. https://llnl.gov/casc/hypre, https://github.com/hypre-space/hypre

  14. [14]

    Cumulated gain -based evalua- tion of ir techniques

    Kalervo Järvelin and Jaana Kekäläinen. Cumulated gain -based evalua- tion of ir techniques. ACM Transactions on Information Systems (TOIS) , 20(4):422–446, 2002

    2002

  15. [15]

    Performance-based numerical solver selection in the light house framework

    Elizabeth Jessup, Pate Motter, Boyana Norris, and Kani ka Sood. Performance-based numerical solver selection in the light house framework. SIAM Journal on Scientific Computing , 38(5):S750–S771, 2016

    2016

  16. [16]

    Feature selection w ith the boruta package

    Miron B Kursa and Witold R Rudnicki. Feature selection w ith the boruta package. Journal of statistical software , 36:1–13, 2010

    2010

  17. [17]

    Computing information retrieval per- formance measures efficiently in the presence of tied scores

    Frank McSherry and Marc Najork. Computing information retrieval per- formance measures efficiently in the presence of tied scores. In European conference on information retrieval , pages 414–421. Springer, 2008

    2008

  18. [18]

    Distributed representations of words and phrases and their compositional- ity

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado , and Jeff Dean. Distributed representations of words and phrases and their compositional- ity. Advances in neural information processing systems , 26, 2013

    2013

  19. [19]

    Preconditioners f or krylov subspace methods: An overview

    John W Pearson and Jennifer Pestana. Preconditioners f or krylov subspace methods: An overview. GAMM-Mitteilungen, 43(4):e202000015, 2020. 15

    2020

  20. [20]

    Glove: Global vectors for word representation

    Jeffrey Pennington, Richard Socher, and Christopher D M anning. Glove: Global vectors for word representation. In Proceedings of the 2014 confer- ence on empirical methods in natural language processing (EM NLP), pages 1532–1543, 2014

    2014

  21. [21]

    The algorithm selection problem

    John R Rice. The algorithm selection problem. In Advances in computers , volume 15, pages 65–118. Elsevier, 1976

    1976

  22. [22]

    Iterative methods for sparse linear systems

    Yousef Saad. Iterative methods for sparse linear systems . SIAM, 2003

    2003

  23. [23]

    Iterative solver selection techniques for sparse linear syst ems

    Kanika Sood. Iterative solver selection techniques for sparse linear syst ems. PhD thesis, University of Oregon, 2019

    2019

  24. [24]

    Graph neural netw orks for se- lection of preconditioners and krylov solvers

    Ziyuan Tang, Hong Zhang, and Jie Chen. Graph neural netw orks for se- lection of preconditioners and krylov solvers. In NeurIPS 2022 Workshop: New Frontiers in Graph Learning , 2022

    2022

  25. [25]

    Mining multi- label data

    Grigorios Tsoumakas, Ioannis Katakis, and Ioannis Vla havas. Mining multi- label data. Data mining and knowledge discovery handbook , pages 667–685, 2010

    2010

  26. [26]

    A theo- retical analysis of ndcg type ranking measures

    Yining Wang, Liwei Wang, Yuanzhi Li, Di He, and Tie-Yan L iu. A theo- retical analysis of ndcg type ranking measures. In Conference on learning theory, pages 25–54. PMLR, 2013

    2013

  27. [27]

    Automatically tun ed linear algebra software

    R Clinton Whaley and Jack J Dongarra. Automatically tun ed linear algebra software. In SC’98: Proceedings of the 1998 ACM/IEEE conference on Supercomputing, pages 38–38. IEEE, 1998

    1998

  28. [28]

    A survey of accelerating parallel sparse linear al gebra

    Guoqing Xiao, Chuanghui Yin, Tao Zhou, Xueqi Li, Yuedan Chen, and Kenli Li. A survey of accelerating parallel sparse linear al gebra. ACM Computing Surveys , 56(1):1–38, 2023

    2023

  29. [29]

    Mm-autosolver: A multimodal machine learning met hod for the auto-selection of iterative solvers and preconditioners

    Hantao Xiong, Wangdong Yang, Weiqing He, Shengle Lin, K eqin Li, and Kenli Li. Mm-autosolver: A multimodal machine learning met hod for the auto-selection of iterative solvers and preconditioners. Journal of Parallel and Distributed Computing , page 105144, 2025

    2025

  30. [30]

    Data-driven performance modeli ng of linear solvers for sparse matrices

    Jae-Seung Yeom, Jayaraman J Thiagarajan, Abhinav Bhat ele, Greg Bron- evetsky, and Tzanio Kolev. Data-driven performance modeli ng of linear solvers for sparse matrices. In 2016 7th International Workshop on Per- formance Modeling, Benchmarking and Simulation of High Perform ance Computer Systems (PMBS) , pages 32–42. IEEE, 2016. 16

    2016