REVIEW 4 major objections 3 minor 31 references
Local Feature Filtering for Scalable and Well-Conditioned Domain-Decomposed Random Feature Methods
T0 review · 4 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Local rank-revealing QR filtering, combined with a sparse preconditioner assembled from the filtered factors, removes most of the ill-conditioning that slows domain-decomposed random feature PDE solvers.
desk verdict Real algorithmic contribution and careful experiments, but the central condition-number theorem is misindexed and false as written—fixable, but needs fixing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is the σ-rank-revealing QR (RRQR) factorization applied locally: each subdomain block $M_j$ is permuted and factored as $M_j \Pi_j = Q_j [R_j; T_j]$ with $R_j$ well-conditioned and the trailing block small, and the columns associated with $R_j$ are retained as the filtered block $\widehat{M}_j = \widehat{Q}_j \widehat{R}_j$. These local factors are then fused into the global sparse right preconditioner $S^{-1} = \sum_j \widehat{W}_j^T \widehat{R}_j^{-1} \widehat{W}_j$, so that the preconditioned least-squares matrix becomes $Q = \widehat{M}S^{-1} = \sum_j V_j^T \widehat{Q}_j \widehat{W}_j$. The key identity is that the $\widehat{R}_j$ factors cancel algebraically, leaving a sum of orthonormal blocks; the conditioning analysis then reduces to bounding the block-tridiagonal overlap matrix $E$ of $Q^T Q = I + E$, whose norm is bounded by $2\alpha$. This is what converts the ill-conditioning problem into a question of how strongly neighbouring subdomains couple.
What would settle it
Compute the spectral condition number $\kappa(\widehat{M}S^{-1})$ for a 2D rectangular decomposition with corner overlaps and compare it with the bound $\sqrt{(1+2\alpha)/(1-2\alpha)}$ using the measured coupling $\alpha = \max_j \|A_j\|_2$; if the measured condition number exceeds the bound, or if the bound blows up ($2\alpha \ge 1$) while LSQR still converges, the strip-geometry assumptions are doing the work. A second check: measure $\|\widehat{Q}_j^T \widehat{Q}_j - I\|_2$ after filtering; if it is not small, the orthonormality premise of Theorem 3.2 is violated.
Extended reading notes
Core claim
The central claim is that the ill-conditioning of the ELM-FBPINN least-squares system has two separable causes — redundancy of random features within each subdomain and correlation across overlapping subdomains — and that a σ-RRQR factorization of each local block $M_j$ addresses both at once. Filtering keeps only columns whose singular values lie above a relative threshold σ, while the upper-triangular factor $\widehat{R}_j$ of the filtered block becomes the building block of a right preconditioner $S^{-1} = \sum_j \widehat{W}_j^T \widehat{R}_j^{-1} \widehat{W}_j$. After preconditioning, the global matrix reduces to $Q = \sum_j V_j^T \widehat{Q}_j \widehat{W}_j$, a sum of orthonormal blocks whose Gram matrix is the identity plus a sparse overlap-coupling matrix. For a simplified strip geometry, Theorem 3.2 bounds the condition number by $\sqrt{(1+2\alpha)/(1-2\alpha)}$ whenever the maximum overlap coupling $\alpha = \max_j \|A_j\|_2$ satisfies $2\alpha < 1$, showing that conditioning depends only on nearest-neighbour overlap strength and not on the number of subdomains or total features. The numerical experiments support the practical claim that this strategy consistently improves conditioning and solver speed while matching or exceeding the accuracy of unpreconditioned and additive-Schwarz baselines.
Load-bearing premise
The deterministic conditioning bound is proven only for subdomains arranged as consecutive 1D strips with nearest-neighbour overlap and exactly orthonormal filtered columns, whereas the headline 2D and (2+1)D experiments use regular rectangular decompositions with corner overlaps, where each block couples to more than two neighbours and post-filter orthogonality is only approximate.
Editorial extensions
If this is right
- Within the strip-decomposition regime covered by Theorem 3.2, the condition number of the preconditioned system stays bounded as the number of subdomains grows, as long as the overlap coupling stays below the 1/2 threshold; this is what makes the method scalable in the 1D strong and weak scaling tests.
- Aggressive filtering (larger σ) monotonically improves conditioning but eventually degrades test accuracy by removing basis functions, so a balance between conditioning and representational capacity is needed; the paper shows the trade-off in its ablation tables.
- Because the preconditioner is assembled from triangular factors and preserves block sparsity, it can be applied by cheap back-substitution without ever forming the normal equations or a dense Gram matrix, avoiding the squared condition number and memory blow-up of normal-equation preconditioning.
- The method matches or beats the additive-Schwarz-preconditioned baseline in accuracy while being roughly an order of magnitude faster and less memory-hungry across the 1D, 2D, and (2+1)D test problems.
Reading between the lines
- If the overlap-coupling constant $\alpha$ is the true controlling quantity, then an adaptive strategy that increases filtering exactly in the overlap regions—rather than a uniform threshold per subdomain—should improve conditioning further; this is not explored in the paper.
- The bound's failure mode for 2D corner overlaps suggests that including diagonal overlap interactions in the preconditioner, as the paper itself flags for future work, could restore the bound in higher dimensions; a testable extension is to add the corner blocks to $S^{-1}$ and measure whether $\kappa(Q)$ drops back toward the predicted value.
- In the random feature setting, the same local RRQR filtering could double as an interpretability tool: the retained columns are actual basis functions rather than linear combinations, so filtering also selects which features carry the solution; the paper notes the interpretability but does not pursue it.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes a local sigma-RRQR filtering and right-preconditioning strategy for domain-decomposed random-feature least-squares systems arising in ELM-FBPINNs. It defines a filtered global matrix bM and a sparse right preconditioner S^{-1} assembled from local triangular factors, shows that the preconditioned matrix is a sum of local orthogonal factors (Proposition 3.1), and derives a deterministic condition-number bound for overlapping strip-like decompositions (Theorem 3.2), with probabilistic refinements in Appendix A. Numerical experiments on a 1D harmonic oscillator, a 2D multi-scale Laplacian, and a (2+1)D wave equation report reductions in condition number by up to eleven orders of magnitude, LSQR speedups of 10 to 1000 times, and improved accuracy versus unpreconditioned, CG, and additive-Schwarz baselines.
Significance. If the results hold, the paper would remove a key obstacle for domain-decomposed random-feature PDE solvers: overlap- and redundancy-induced ill-conditioning. The experimental methodology is a strength: five seeds with reported ranges, ablations over sigma, activation, and depth, strong and weak scaling studies, and comparisons against exact solutions and a high-fidelity finite-difference solver. The proposed preconditioner is attractive because it is sparse, avoids forming normal equations, and is assembled only from local RRQR factors. However, the central deterministic bound is misstated as written and covers only a simplified 1D strip geometry, so the theoretical support for the headline 2D and (2+1)D claims is currently incomplete.
major comments (4)
- [Section 3.4, Eq. (8) and Theorem 3.2] The off-diagonal block A_j is defined as Q_{j+1,j}^T Q_{j,j}, but the (j,j+1) block of Q^T Q is Q_{j+1,j}^T Q_{j+1,j+1}. With the printed definition, alpha is not the adjacent-subdomain coupling, and the theorem's conclusion is false: for J=2, K=1, Q11=sqrt(1-b^2), Q21=b, Q22=1, and b=0.99, the printed alpha is about 0.0995, so 2alpha<1 holds, but the actual kappa(Q)=sqrt((1+b)/(1-b)) is about 14.1, far above the claimed bound. The proof goes through with A_j:=Q_{j+1,j}^T Q_{j+1,j+1}; please correct the definition, the interpretation of alpha as overlap strength, and Remark 3.4, and re-examine the claim that RRQR filtering reduces this quantity.
- [Section 3.4 vs Section 4] The deterministic bound is derived only for consecutive 1D strips with nearest-neighbor overlap and exactly orthonormal blocks, but the main 2D and (2+1)D experiments use rectangular decompositions with corner overlaps, where each block couples to more than two neighbors and post-filter orthogonality is approximate. The paper's own discussion in Section 4.4.3 concedes that the preconditioner neglects overlapping contributions between subdomains in higher dimensions. Therefore Theorem 3.2 does not justify the condition numbers reported for the 2D and (2+1)D experiments; the abstract and introduction should state this limitation explicitly, and the paper should indicate which of the numerical claims are covered by the theorem.
- [Appendix A.2] The probabilistic estimate is derived under a Haar-distributed semi-orthogonal model for the filtered blocks that is asserted but not validated. Figure A.20 only shows that measured condition numbers grow with K and delta; it does not compare measured kappa(Q) against the upper bound (A.3), nor does it test whether the block model is accurate. Consequently, the label 'Numerical validation of bound' overstates the evidence. Either provide a quantitative comparison or present Figure A.20 as a qualitative trend only.
- [Section 3.4 and Algorithm 1] The theorem assumes each subdomain contributes exactly K features, but after RRQR filtering each subdomain retains a different number k_j of columns. The block structure in Eq. (8) and the definition of Q therefore need to be restated with variable block widths, and the condition-number argument must be checked for that case. The proof is likely to survive because it only needs per-block values alpha_j, but the statement as written is not the system actually being solved.
minor comments (3)
- [Section 3.4] The first sentence refers to 'Theorem 3.1', but the result being used is Proposition 3.1; please correct the reference.
- [Section 4.4.1] The text 'the FBPINN and PINN baselines are and nearly an order of magnitude slower' contains a missing word; it should read something like 'are also nearly an order of magnitude slower'.
- [Section 3.2 and Algorithm 2] The filtered selection matrices \hat W_j are not defined after column dropping; their dimensions should be stated explicitly (for example, \hat W_j in R^{k_j x JK}) so that Proposition 3.1 and Algorithm 2 are unambiguous.
Circularity Check
No significant circularity: the conditioning analysis is a genuine mathematical derivation from the RRQR factors, the reported gains are measured against external benchmarks, and the self-citations are background only.
full rationale
The paper's central derivation chain is not circular. The preconditioner S^{-1} = sum_j bW_j^T bR_j^{-1} bW_j is constructed from the local RRQR factors, and the identity Q = bM S^{-1} = sum_j V_j^T bQ_j bW_j follows by algebra (Proposition 3.1). Theorem 3.2 then proves a bound on kappa(Q) in terms of the overlap coupling alpha; this is a theorem about the constructed Q, not a quantity fitted to the experimental outcomes. The reported condition-number reductions and LSQR speedups are measurements on the actual filtered systems (Tables 1-3), and accuracy is checked against exact solutions and an independent finite-difference solver, so the empirical claims do not presuppose the theory. The only self-referential elements are background citations to the authors' earlier ELM-FBPINN/FBPINN work, which are not load-bearing for the new preconditioning result. Appendix A's model of filtered blocks as random semi-orthogonal matrices is an explicit modeling assumption used to obtain a probabilistic estimate; it is not used to fit the deterministic bound or the experimental results, so it does not make the predictions equivalent to their inputs. A possible indexing error in the displayed A_j in Theorem 3.2 would be a correctness concern, not a circularity one, and is not treated as circularity here.
Assumptions & free parameters
free parameters (3)
- sigma: RRQR relative threshold =
1e-8 in baseline runs; 1e-10 to 1e-2 in ablations
- delta: subdomain overlap ratio =
2.9 in baseline runs; 1.45 to 23.2 in scaling studies
- lambda_l: boundary-condition loss weights =
not reported
assumptions (4)
- standard math Existence and guarantees of sigma-RRQR factorizations (Gu-Eisenstat strong RRQR)
- domain assumption The randomized local feature ansatz can approximate the PDE solution before filtering
- ad hoc to paper Haar-distributed semi-orthogonal model for filtered feature blocks
- domain assumption Strip-wise nearest-neighbor overlap block structure
Cite this review
Pith. "Pith review of Local Feature Filtering for Scalable and Well-Conditioned Domain-Decomposed Random Feature Methods." pith.science (2026). https://pith.science/paper/VCVM46SB
@misc{pith2026250617626,
author = {Pith},
title = {Pith review of: Local Feature Filtering for Scalable and Well-Conditioned Domain-Decomposed Random Feature Methods},
year = {2026},
howpublished = {\url{https://pith.science/paper/VCVM46SB}},
note = {Machine review of arXiv:2506.17626}
}
read the original abstract
Random Feature Methods (RFMs) and their variants such as extreme learning machine finite-basis physics-informed neural networks (ELM-FBPINNs) offer a scalable approach for solving partial differential equations (PDEs) by using localized, overlapping and randomly initialized neural network basis functions to approximate the PDE solution and training them to minimize PDE residuals through solving structured least-squares problems. This combination leverages the approximation power of randomized neural networks and the parallelism of domain decomposition. However, the resulting least-squares systems are often severely ill-conditioned, due to local redundancy among random basis functions, which significantly affects the convergence of standard solvers. In this work, we introduce a block rank-revealing QR (RRQR) filtering and preconditioning strategy that operates directly on the structured least-squares problem. First, local RRQR factorizations identify and remove redundant basis functions while preserving numerically informative ones, reducing problem size, and improving conditioning. Second, we use these factorizations to construct a right preconditioner for the global problem which preserves block-sparsity and numerical stability. Third, we derive deterministic bounds of the condition number of the preconditioned system, with probabilistic refinements for small overlaps. We validate our approach on challenging, multi-scale PDE problems in 1D, 2D, and (2+1)D, demonstrating reductions in condition numbers by up to eleven orders of magnitude, LSQR convergence speedups by factors of 10-1000, and higher accuracy than both unpreconditioned and additive Schwarz-preconditioned baselines, all at significantly lower memory and computational cost. These results establish RRQR-based preconditioning as a scalable, accurate, and efficient enhancement for RFM-based PDE solvers.
Figures
Figures from the paper (18 more)
Reference graph
Works this paper leans on
- [20]
-
[1]
J. Chen, X. Chi, W. E, Z. Yang, Bridging traditional and machine learning-based algorithms for solving pdes: the random feature method, J Mach Learn 1 (2022) 268–98
work page 2022
-
[2]
S. Anderson, V . Dolean, B. Moseley, J. Pestana, ELM-FBPINN: e fficient finite-basis physics-informed neural networks, arXiv preprint arXiv:2409.01949 (2024)
arXiv 2024
-
[3]
I. E. Lagaris, A. Likas, D. I. Fotiadis, Artificial neural networks for solving ordinary and partial differential equations, IEEE Transactions on Neural Networks 9 (5) (1998) 987–1000. arXiv:9705023, doi:10.1109/72.712178
-
[4]
M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial di fferential equations, Journal of Computational Physics 378 (2019) 686–707. doi:10. 1016/j.jcp.2018.10.045. URL https://doi.org/10.1016/j.jcp.2018.10.045
-
[5]
M. G. Dissanayake, N. Phan-Thien, Neural-network-based approximations for solving partial di fferential equations, Communications in Numerical Methods in Engineering 10 (3) (1994) 195–201
work page 1994
-
[6]
N. Rahaman, A. Baratin, D. Arpit, F. Draxlcr, M. Lin, F. A. Hamprecht, Y . Bengio, A. Courville, On the spectral bias of neural networks, in: 36th International Conference on Machine Learning, ICML 2019, V ol. 2019-June, International Machine Learning Society (IMLS), 2019, pp. 9230–9239. arXiv:1806.08734. URL http://arxiv.org/abs/1806.08734
arXiv 2019
-
[7]
B. Moseley, A. Markham, T. Nissen-Meyer, Finite basis physics-informed neural networks (FBPINNs): a scalable domain decomposition approach for solving di fferential equations, Advances in Computational Mathematics 2023 49:4 49 (4) (2023) 1–39. doi:10.1007/ S10444-023-10065-9 . URL https://link.springer.com/article/10.1007/s10444-023-10065-9
Show all 31 references
-
[8]
S. Wang, H. Wang, P. Perdikaris, On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks, Computer Methods in Applied Mechanics and Engineering 384 (2021) 113938.arXiv:2012.10047, doi:10.1016/j.cma....
2021 arXiv
-
[9]
Moseley, Physics-informed machine learning: from concepts to real-world applications, Ph.D
B. Moseley, Physics-informed machine learning: from concepts to real-world applications, Ph.D. thesis, University of Oxford (2022). doi:10.13039/501100000266
2022 doi
-
[10]
A. D. Jagtap, E. Kharazmi, G. E. Karniadakis, Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems, Computer Methods in Applied Mechanics and Engineering 365 (2020) 113028
2020
-
[11]
A. D. Jagtap, G. E. Karniadakis, Extended physics-informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations, Communications in Computational Physics 28 (5) (2020)
2020
-
[12]
Kharazmi, Z
E. Kharazmi, Z. Zhang, G. E. Karniadakis, hp-VPINNs: Variational physics-informed neural networks with domain decomposition, Computer Methods in Applied Mechanics and Engineering 374 (2021) 113547
2021
-
[13]
W. Li, X. Xiang, Y . Xu, Deep domain decomposition method: Elliptic problems, in: Mathematical and Scientific Machine Learning, PMLR, 2020, pp. 269–286
2020
-
[14]
Dolean, A
V . Dolean, A. Heinlein, S. Mishra, B. Moseley, Finite basis physics-informed neural networks as a Schwarz domain decomposition method, in: International Conference on Domain Decomposition Methods, Springer, 2022, pp. 165–172
2022
-
[15]
Dolean, A
V . Dolean, A. Heinlein, S. Mishra, B. Moseley, Multilevel domain decomposition-based architectures for physics-informed neural net- works, Computer Methods in Applied Mechanics and Engineering 429 (2024) 117116
2024
-
[16]
G. B. Huang, Q. Y . Zhu, C. K. Siew, Extreme learning machine: Theory and applications, Neurocomputing 70 (1-3) (2006) 489–501. doi:10.1016/j.neucom.2005.12.126
2006 doi
-
[17]
Y .-H. Pao, Y . Takefuji, Functional-link net computing: theory, system architecture, and functionalities, Computer 25 (5) (1992) 76–79
1992
-
[18]
Lukoševi ˇcius, H
M. Lukoševi ˇcius, H. Jaeger, Reservoir computing approaches to recurrent neural network training, Computer Science Review 3 (3) (2009) 127–149. doi:https://doi.org/10.1016/j.cosrev.2009.03.005. URL https://www.sciencedirect.com/science/article/pii/S1574013709000173
2009 doi
-
[19]
Datar, T
C. Datar, T. Kapoor, A. Chandra, Q. Sun, I. Burak, E. L. Bolager, A. Veselovska, M. Fornasier, F. Dietrich, Solving partial di fferential equations with sampled neural networks (2024). arXiv:2405.20836. URL https://arxiv.org/abs/2405.20836
2024 arXiv
-
[21]
T. F. Chan, Rank revealing qr factorizations, Linear Algebra and its Applications 88–89 (1987) 67–82. doi:10.1016/0024-3795(87) 90103-0
1987 doi
-
[22]
Rahimi, B
A. Rahimi, B. Brecht, Random features for large-scale kernel machines, Advances in Neural Information Processing Systems (2007)
2007
-
[23]
M. Gu, S. C. Eisenstat, E fficient algorithms for computing a strong rank-revealing qr factorization, SIAM Journal on Scientific Computing 17 (4) (1996) 848–869. doi:10.1137/0917055
1996 doi
-
[24]
C. C. Paige, M. A. Saunders, LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares, ACM Transactions on Mathematical Software (TOMS) 8 (1) (1982) 43–71. doi:10.1145/355984.355989/ASSET/ CF3C8EE0-D698-4E67-BD8A-BF0B25681EEC/ASSETS/355984.355989.FP.PNG . URL ht...
1982
-
[25]
Bradbury, R
J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, Q. Zhang, JAX: composable transformations of Python+NumPy programs (2018). URL http://github.com/jax-ml/jax
2018
-
[26]
Virtanen, R
P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, ˙I. Polat, Y . Fen...
2020
-
[27]
G. W. Stewart, The e fficient generation of random orthogonal matrices with an application to condition estimators, SIAM Journal on Numerical Analysis 17 (3) (1980) 403–409. doi:10.1137/0717034
1980 doi
-
[28]
J. A. Tropp, A comparison principle for functions of a uniformly random subspace, Probability Theory and Related Fields 153 (3–4) (2011) 759–769. doi:10.1007/s00440-011-0360-9
2011 doi
-
[29]
Diaconis, M
P. Diaconis, M. Shahshahani, The subgroup algorithm for generating uniform random variables, Probability in the Engineering and Informational Sciences 1 (1) (1987) 15–32. doi:10.1017/S0269964800000255
1987 doi
-
[30]
M. W. Meckes, The Random Matrix Theory of the Classical Compact Groups, V ol. 218 of Cambridge Tracts in Mathematics, Cambridge University Press, 2019. doi:10.1017/9781108642324. URL https://doi.org/10.1017/9781108642324
2019 doi
-
[31]
Komatitsch, R
D. Komatitsch, R. Martin, An unsplit convolutional perfectly matched layer improved at grazing incidence for the seismic wave equation, Geophysics 72 (5) (2007) SM155–SM167. 29
2007
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.