Latent Mamba Operator for Partial Differential Equations

Karn Tiwari; Niladri Dutta; N M Anoop Krishnan; Prathosh A P

REVIEW 3 major objections 5 minor 3 cited by

Latent Mamba Operator for Partial Differential Equations

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The Latent Mamba Operator claims its state-space layer is a learnable kernel integral and reports the best accuracy on all six benchmark PDEs it tests, a 32.3% average gain over prior models.

desk verdict LaMO is a strong empirical entry in SSM-based operator learning; the theory appendix and one baseline comparison need work, but the core results look real. read the letter →

arxiv 2505.19105 v2 pith:U5AGS5UI submitted 2025-05-25 cs.LG

Karn Tiwari , Niladri Dutta , N M Anoop Krishnan , Prathosh A P This is my paper

classification cs.LG MSC 68T07

keywords neuraloperatorpartialdifferentialequationsstate-spacemodelsMambalatentspacekernelintegrallearningscientificmachine

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that state-space models, the linear-time sequence models behind Mamba, can act as the core of a neural operator: a network that learns the map from a PDE's inputs (coefficients, geometry, boundary conditions) to its solution field. Its Latent Mamba Operator (LaMO) compresses the mesh or grid into a small set of latent tokens and then runs a bidirectional state-space scan over those tokens, keeping cost linear in the number of mesh points. On six standard PDE benchmarks spanning fluid and solid physics over regular grids, structured meshes, and point clouds, LaMO reports the lowest mean relative $\ell^2$ error in every case, an average 32.3% improvement over the best prior baseline. The paper also proves a theoretical identity for the architecture: a latent state-space layer approximates the integral operator $G(a)(x) = \int_\Omega \kappa(x, y) a(y)\, dy$, placing the SSM in the same kernel-integral framework as Fourier and attention-based operators. If both claims hold, SSM architectures become a drop-in alternative to attention for operator learning, with linear scaling and competitive or better accuracy.

What carries the argument

The object that carries the argument is the latent-SSM layer: a selective state-space scan applied in multiple directions over compressed latent tokens rather than over physical mesh points. In continuous time the scan is the linear ODE $h'(t) = A h(t) + B x(t)$, $y(t) = C h(t)$; zero-order-hold discretization turns it into the recurrence $h[k] = \bar{A} h[k-1] + \bar{B} x[k]$, and with input-dependent parameters, as in Mamba, the whole layer acts as a data-dependent convolution kernel over the token sequence. The paper's Theorem 3.4 identifies this scan with a Monte-Carlo approximation of the kernel integral $G(a)(x) = \int_\Omega \kappa(x, y) a(y)\, dy$, the same integral-operator formalism used to justify Fourier and attention neural operators; the Perceiver-style encoder-decoder around it reduces the token count from $N$ to $M$, making the per-layer cost $O(NMD + MD)$, hence linear in the mesh size.

What would settle it

Re-run the strongest baselines, in particular Transolver, from their official code under exactly LaMO's per-benchmark protocol, same loss including the Darcy spatial-gradient term, same 500-epoch AdamW/OneCycleLR schedule, and a matched parameter budget, and compare mean relative $\ell^2$ errors; if the reported per-benchmark margins of 17.4% to 51.9% shrink to within the paper's own standard deviations, the state-of-the-art claim fails. A separate decisive check would compare the learned SSM kernel $\kappa(x, y)$ on Darcy flow against the analytic Green's function of $-\nabla \cdot (a \nabla u) = f$ to test Theorem 3.4 directly.

Watch

Extended reading notes

Core claim

The central claim is that a neural operator built from a latent state-space model recovers the solution map of parametric PDEs more accurately and more cheaply than attention-based operators. Architecturally, LaMO composes a lifting operator $P$, a Perceiver-style encoder $E$ that projects $N$ physical tokens into $M \ll N$ input-dependent latent tokens, $L$ latent blocks each applying a multi-head bidirectional (or four-directional, on regular grids) selective SSM as the token-mixing layer, a decoder $D$, and a projection $Q$: $G_\theta = Q \circ D \circ \mathcal{L}_L \circ \cdots \circ \mathcal{L}_1 \circ E \circ P$. The theoretical claim, Theorem 3.4 (proved in the appendix as Theorem B.12), is that a latent-SSM layer approximates the integral operator $G(a)(x) = \int_\Omega \kappa(x, y) a(y)\, dy$, so the SSM recurrence with its input-dependent transition matrices is a Monte-Carlo realization of a learnable kernel integral; the paper further notes that this kernel form $C(x)\operatorname{Diag}(\prod A) B(y)$ structurally matches the orthogonal-attention kernel of ONO without needing its orthonormalization. The empirical claim is that across Darcy flow, turbulent Navier-Stokes, Elasticity, Plasticity, Airfoil, and Pipe benchmarks, LaMO achieves the lowest mean relative $\ell^2$ error on every dataset, improving on the second-best operator by margins from 17.4% to 51.9% per benchmark while using fewer parameters and linear-time scans.

Load-bearing premise

The state-of-the-art claim rests on the premise that the published baseline scores are directly comparable to LaMO's; most of those scores were quoted from the baselines' own papers rather than re-run under LaMO's exact protocol, which adds a spatial-gradient smoothing term to the Darcy training loss that the baselines were not given.

Editorial extensions

If this is right

Attention can be replaced by state-space scans as the kernel-integral engine of a neural operator without sacrificing accuracy, because the SSM layer is shown to implement the same integral-operator form.
Operator inference scales linearly with mesh resolution rather than quadratically, since the latent token count is fixed and the scan cost is linear in that count.
Non-causal processing is needed for PDE solving: the paper's ablation shows that a unidirectional scan degrades accuracy, confirming that forward-and-backward information flow matters.
The architecture is data-efficient, reaching the second-best operator's accuracy with roughly 40% of the Darcy and Navier-Stokes training data and 20% of the Plasticity data.
The reported gains appear on every geometry type, regular grids, structured meshes, and point clouds, so the approach is not tied to a single discretization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Under the paper's Theorem 3.4 reading, the latent-token count $M$ is effectively the quadrature resolution of the kernel integral; the non-monotonic accuracy curve on Airfoil, best near 128-256 tokens and worse at 512, then predicts that the optimal $M$ tracks the smallest physical feature scale of the solution, a check the paper does not run.
Because the encoder derives latent tokens from the input rather than from a fixed grid, it can serve as a resolution-agnostic geometric tokenizer; a natural next step, which the authors list only as future work, is to pretrain one latent SSM stack across several PDE families as a foundation model.
The learned transition products $\prod A_k$ form an input-dependent positional mask, which plausibly lets the model adapt its correlation length to local physics, short near discontinuities and long in smooth regions; measuring these products on Darcy versus Navier-Stokes would test whether that adaptive masking drives the reported gains.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read LaMO. The architecture is the real contribution: latent tokens plus bidirectional/multidirectional SSM scanning is a natural and well-executed follow-on to Transolver and Perceiver, and the efficiency numbers support the linear-complexity claim. Six standard benchmarks, relative L2 errors lower than Transolver across the board, with standard deviations on LaMO and Transolver runs. That is an honest empirical package and it deserves a serious referee.

The main concern is the theory. Appendix B is not in a publishable state. Lemma B.11 asserts a bijection between N grid tokens and M latent tokens while M << N; that is impossible. The proof's argmax construction with a K-block constraint doesn't fix cardinality, and the subsequent unit-determinant Jacobian simplification in Theorem B.12 is unjustified. Without a legitimate change-of-variables, the latent-domain integral equivalence doesn't follow. Lemma B.10 is also close to definitional: the kernel is defined as the SSM product, so concluding the SSM approximates an integral operator with that kernel is a restatement. The theory section should be either corrected to a formal approximation statement with a proper measure or weakened to an observation.

Second, the Darcy benchmark. Table 6 discloses that LaMO trains Darcy with l2 + 0.1*l_gdl, an extra gradient regularization 'following ONO work,' while baseline numbers are taken from official papers/codebases without that regularization. The Darcy margin against Transolver is 33.9%, the largest in the table, and this protocol asymmetry could explain part of it. This is documented, not hidden, but it makes the headline 32.3% average less crisp. A quick ablation of LaMO on Darcy with plain l2 would settle it.

Everything else is in good shape. The ablations on latent tokens, state dimension, scanning directions, and data efficiency are useful. The code link is there. The related work covers MemNO and Hu et al., so the SSM-for-PDE lineage is acknowledged.

My honest take: the empirical core holds up and the architecture is worth knowing. The theory needs a rewrite, and the Darcy comparison needs a control. This is a conditional accept situation, not a reject. Send it to reviewers who care about neural operators, and tell them to focus on the benchmark protocol and the appendix.

Referee Report

3 major / 5 minor

Summary. The paper introduces the Latent Mamba Operator (LaMO), a neural operator that embeds an input PDE field into a small set of latent tokens and processes them with bidirectional/multidirectional state-space models (SSMs). The authors claim three contributions: (1) a scalable architecture with linear complexity in the number of mesh points, (2) a theoretical result (Theorem 3.4) that a latent-SSM layer approximates a kernel integral operator, and (3) state-of-the-art empirical performance on six PDE benchmarks, with an average 32.3% relative-L2 improvement over the second-best baseline. The appendix provides derivations connecting SSMs to Monte Carlo approximation of integral operators, and extensive ablations, efficiency comparisons, and visualizations. The manuscript also makes public code available.

Significance. If the empirical claim is sustained, LaMO would be a meaningful advance: it reports consistently lower errors than strong transformer-based baselines such as Transolver across regular grids, structured meshes, and point clouds, with a favorable parameter count and training time. The reported standard deviations for LaMO and Transolver (Appendix E.2) and the reproducibility-oriented release of code are strengths that support the empirical case. The theoretical connection between SSMs and kernel integrals is conceptually useful, as it links the fast-growing SSM literature to neural operator theory; however, the proof as written is not valid, and the benchmark protocol contains a training-loss asymmetry that affects the headline improvement. These issues are consequential but appear addressable, so the contribution is potentially publishable after substantial revision.

major comments (3)

[Appendix D.2 and D.3; Table 1; Table 6] The headline 32.3% average improvement and the specific Darcy margin (33.9% over Transolver) rest on a training-protocol asymmetry that the manuscript itself documents. Table 6 states that LaMO trains Darcy with loss l2 + 0.1*l_gdl, an additional spatial-gradient regularizer adopted 'following ONO work,' while Section D.3 reports that most baseline numbers, including Transolver, are taken from their original papers or official codebases without retraining under the same loss. Since l_gdl directly penalizes high-frequency prediction error, the Darcy comparison is not apples-to-apples, and the 32.3% average is dominated by this one large margin. To support the SOTA claim, the authors should either (a) ablate LaMO on Darcy under plain relative-L2 loss, or (b) retrain Transolver and other baselines with the identical l2 + 0.1*l_gdl objective, and report both results; until then the central empirical claim is not fully established.
[Appendix B, Lemma B.11] Lemma B.11 asserts a bijection f between the N-element input domain Omega and the M-element latent domain Omega_s, but the construction in Eq. (58)-(59) cannot be bijective when M << N, which is the defining property of the latent encoder (M << N in Eq. (10) and Table 6). The 'not assigned previously' constraint only prevents duplicate assignments; it does not create enough latent tokens to cover N inputs. This invalidates the claim that 'the cardinalities of the two domains are equivalent' and undermines the change-of-variables step in the proof of Theorem B.12 at Eqs. (63)-(66), which relies on a diffeomorphic projection f. The theorem may still be salvageable as an approximation statement, but as written the proof is not mathematically valid.
[Appendix B, Lemma B.10 and Theorem B.12] Lemma B.10 is close to definitional: Eq. (56) defines the kernel parameterization kappa(y_i, y_j) directly as W_C(y_i)(\prod A)W_B(y_j), the exact product of SSM matrices, and then concludes that the SSM is a Monte Carlo approximation of the integral operator with that kernel. This shows that the SSM recurrence can be written as a kernel sum, but it does not establish that the SSM approximates the solution operator G for a PDE; it is a reparameterization of the SSM computation. Theorem 3.4 inherits this limitation, and the additional assumption |det(nabla_yz f^{-1}(yz))| = 1 in Eq. (67) is unjustified and generally false for a nontrivial projection. The authors should either prove a genuine approximation result with explicit error bounds and stated measure-theoretic assumptions, or clearly frame the analysis as a structural analogy rather than an equivalence theorem.

minor comments (5)

[Section 2.2, Eq. (5) and Eq. (4)] The notation uses A and B for both continuous and discrete system matrices; the overline notation introduced in the appendix (\bar{A}, \bar{B}) should be used consistently in the main text to avoid confusion.
[Table 1 caption and Table 8 footnote] The caption of Table 8 refers to 'second-best model' but Table 1 shows that Transolver is not the second-best for every benchmark (e.g., Elasticity, where GNOT is second) unless the table is restricted to the listed comparison; please clarify the selection rule.
[Section 3.1, Remark 3.1] The claim that ViT patches are a special case of latent tokens is plausible but not formalized; a short argument for why Softmax(Linear(X)) can exactly realize patch-wise indicator weights would strengthen the remark.
[Appendix D.2, Table 6] The batch-size row '4 2 1 8 4 4' is ambiguous because it is not clear which value corresponds to which benchmark; align the table entries explicitly with the Darcy, Navier-Stokes, Elasticity, Plasticity, Airfoil, and Pipe columns.
[Throughout] There are several typographical issues, including inconsistent hyphenation in 'HT-NET', missing spaces in 'ONO (Xiao et al., 2023) introduces orthogonal regularization', and the phrase 'We proposeLatent Mamba Neural Operator' in the contributions list; a careful proofread is needed.

Circularity Check

2 steps flagged · score 5.0 of 10

The empirical SOTA claim is benchmarked externally, but the central theoretical result is definitional: Lemma B.10 defines the kernel κ as the SSM product and then concludes the SSM is a Monte Carlo integral operator with that same kernel, so Theorem 3.4 holds by construction.

self definitional [Appendix B, Lemma B.10, Eq. (54)-(57); used by Theorem 3.4]
"In SSM, the integral kernel κ is parameterized as follows: κ(y_i, y_j) = W_C(y_i)(∏_{y_k ≤ y_i} A) W_B(y_j) ... Substituting this parameterization into the Monte Carlo approximation, we obtain ... This expression demonstrates that the SSM operator can be interpreted as a Monte Carlo approximation of the integral operator G, where the state-space dynamics parameterize the kernel κ."

The lemma begins with an arbitrary measurable kernel κ in the definition of G, then sets κ(y_i, y_j) to be exactly the SSM product W_C(y_i)(∏ A) W_B(y_j). With that substitution, the Monte Carlo expression is simply the SSM recurrence Y = M X (compare Appendix Eq. 82, M_{i,j} = C_i ∏ A_k B_j) rewritten in integral notation. The conclusion is therefore an identity by construction, not a derived approximation property. It does not establish that SSMs approximate an independent class of kernel integrals; it renames the SSM's own transition matrix as a 'kernel'.
self definitional [Section 3.2, Theorem 3.4 (proof in Appendix B, Theorem B.12)]
"An latent-SSM layer approximates the integral operator G: L^2(Ω, R^d) → L^2(Ω, R^d), defined as follows: G(a)(x) = ∫_Ω κ(x, y)a(y) dy, where κ: Ω × Ω → R^{d×d} is the kernel function characterizing the operator G."

This theorem is the paper's advertised theoretical connection, but its proof invokes Lemma B.10 for the 'SSM among the latent tokens' step. Since Lemma B.10's κ was defined to be the SSM product itself, the theorem inherits that definitional identity: the latent-SSM layer is shown to compute an integral whose kernel is the SSM's own product. The added latent-isomorphism step (Lemma B.11, cited from Wu et al.) only changes coordinates; it supplies no independent approximation content. Thus the stated equivalence is forced by the definition of κ rather than derived from kernel-integral theory.

full rationale

The headline empirical claim (32.3% average improvement, Table 1) is compared against external baselines and is not circular in the definitional sense. The Darcy training-protocol asymmetry disclosed in Appendix D.2/D.3 (LaMO uses l2 + 0.1 l_gdl following ONO, while most baseline numbers are taken from official papers without that regularizer) is an experimental-comparability concern, not a circularity, so it does not raise the circularity score. The concrete circular step is the theoretical derivation: Lemma B.10 defines the integral kernel κ(y_i, y_j) as the SSM product W_C(y_i)(∏ A) W_B(y_j), then concludes that the SSM is a Monte Carlo approximation of the integral operator with that same kernel. The conclusion is the definition. Theorem 3.4, listed as a main contribution, inherits this by-construction character. The self-citations in the paper (CoNO, ConoAir) are related-work mentions and are not load-bearing. Overall, the empirical contribution stands independently, but the claimed theoretical equivalence is a renaming of the SSM recurrence, making the paper partially circular rather than fully so.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The theoretical equivalence relies on uniform Monte Carlo quadrature, invertibility of A, a diagonal A, an isomorphism between latent and physical domains that is asserted despite differing cardinalities, and a unit Jacobian determinant. The last two are the most fragile and are not proven. The empirical result depends on several hand-set hyperparameters, notably the latent token count and the Darcy gradient-loss weight.

free parameters (7)

Latent token count M = 1936 (Darcy), 1024 (Navier-Stokes), 64 (Elasticity, Plasticity, Airfoil, Pipe)
Main architectural bottleneck; ablations in Tables 2 and 3 show error varies strongly with M.
SSM state dimension DState = 64
Fixed state dimension of the selective SSM; ablation in Table 2 shows 64 gives the best Darcy error among tested values.
SSM expand dimension = 2
Internal expansion factor of the SSM block; chosen after the ablation in Table 2.
Number of SSM heads = 1 (Darcy), 4 (others)
Multi-head SSM splits the embedding dimension; set per benchmark in Table 6 without reported tuning curves.
Number of layers = 8
Depth of the LaMO stack; Figure 2(c) shows 8 layers is the tested maximum, with performance still improving.
Embedding dimension = 64 (Darcy), 256 (Navier-Stokes), 128 (others)
Channel width per benchmark; only three values are used across all datasets.
Darcy gradient regularization weight = 0.1 on l_gdl, added to l2 loss
Applied only to Darcy following ONO; baselines are not trained with this term, so the Darcy margin may not be apples-to-apples.

assumptions (6)

standard math Variation of constants formula for linear ODE h'(t)=Ah(t)+Bx(t), giving h(t)=e^{A(t-t0)}h(t0)+∫ e^{A(t-s)}Bx(s)ds
Used in Lemma B.1 to express the SSM solution as a convolution; the paper cites Williams et al. (2007) for this.
domain assumption Uniformly sampled grid points approximate integrals over Ω by Monte Carlo quadrature with weight |Ω|/N
Lemma B.10 uses this to replace ∫ κ(x,y)u(y)dμ(y) with an unweighted average over N mesh points; this bridges continuous kernel integrals and discrete SSM scans.
ad hoc to paper The latent token domain Ω_s is isomorphic to the full grid domain Ω even though M << N
Lemma B.11 asserts a bijective projection from N physical tokens to M latent tokens with M << N, which is impossible by cardinality; the proof is internally inconsistent and the claim is load-bearing for Theorem B.12.
domain assumption The continuous system matrix A is invertible, so B = A^{-1}(e^{ΔA}-I)B
Proposition B.4 requires invertibility to derive the discrete B; the implementation instead uses the approximate form B ≈ ΔB, so the exact formula is not what is trained.
ad hoc to paper The Jacobian determinant |det(∇f^{-1})| is taken to be 1
Theorem B.12 states 'for simplicity, we take |det(∇ f^{-1})|=1'; this normalization is asserted without justification and is needed for the latent-space change of variables.
domain assumption Mamba's A matrix is diagonal, A=Diag(λ_1,...,λ_p)
Remark 2.1 assumes the diagonal structure of A; it is required for the scalar product form of the kernel in Lemma B.10.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Latent Mamba Operator for Partial Differential Equations." pith.science (2026). https://pith.science/paper/U5AGS5UI

@misc{pith2026250519105,
  author       = {Pith},
  title        = {Pith review of: Latent Mamba Operator for Partial Differential Equations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U5AGS5UI}},
  note         = {Machine review of arXiv:2505.19105}
}

read the original abstract

Neural operators have emerged as powerful data-driven frameworks for solving Partial Differential Equations (PDEs), offering significant speedups over numerical methods. However, existing neural operators struggle with scalability in high-dimensional spaces, incur high computational costs, and face challenges in capturing continuous and long-range dependencies in PDE dynamics. To address these limitations, we introduce the Latent Mamba Operator (LaMO), which integrates the efficiency of state-space models (SSMs) in latent space with the expressive power of kernel integral formulations in neural operators. We also establish a theoretical connection between state-space models (SSMs) and the kernel integral of neural operators. Extensive experiments across diverse PDE benchmarks on regular grids, structured meshes, and point clouds covering solid and fluid physics datasets, LaMOs achieve consistent state-of-the-art (SOTA) performance, with a 32.3% improvement over existing baselines in solution operator approximation, highlighting its efficacy in modeling complex PDE solutions.

Figures

Figures reproduced from arXiv: 2505.19105 by the authors.

**Figure 1.** Overview. (1) The input function a(x) is lifted to a higher-dimensional representation using the lifting operator P. (2) The encoder E maps the input from the physical domain to the latent domain, where the latent block (Bottom Left) performs the kernel integral via SSMs, applies channel mixing and decodes the latent tokens back to the physical domain using the decoder D. (3) A multi-headed bidirectional SSM (Bottom… view at source ↗

**Figure 2.** Model performance on the scalability of the Darcy Flow benchmark evaluated across various aspects: (Left) Data Efficiency, measuring performance with varying amounts of training data; (Middle Left) Resolution, assessing the impact of different input spatial resolutions; (Middle Right) Model Depth, analyzing performance with increasing layers; and (Right) Embedding Dimension, examining the effect of varying latent sp… view at source ↗

**Figure 3.** Visual Comparison. The (Top Row) displays the ground truth, the (Middle Row) presents the error heatmap of Transolver, and the (Bottom Row) presents the error heatmap for LaMO on (a) Darcy Flow, (b) Plasticity, and (c) Navier-Stokes benchmark. LaMO achieves a relative l2 error improvement of 75% (0.0454 → 0.0117) compared to Transolver. Thus validating it as consistent across the turbulent regions. Latent Tokens: U… view at source ↗

Figures from the paper (10 more)

**Figure 4.** Figure 4: Efficiency comparison of top five baselines on (a) Darcy and (b) Airfoil benchmark per epoch, respectively. Efficiency: To further analyze the performance of the proposed model, we present its efficiency metrics in [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: The diagram presents an overview of the Neural Operator learning task benchmark PDEs classified as Fluid Physics and Solid Physics. (Top Row) showcases three specific PDEs: Darcy Flow, Airfoil, and Elasticity. (Bottom Row), an additional set of three PDEs is shown: Nav…

**Figure 6.** Figure 6: Visualization of multidirectional scan for regular grid benchmark dataset. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]

**Figure 7.** Figure 7: Efficiency comparison between LaMO and Transolver (Left) Training Time, (Middle) Inference Time and (Right) Memory consumption per epoch on all the benchmark dataset. E.4. Time Complexity [PITH_FULL_IMAGE:figures/full_fig_p026_7.png]

**Figure 8.** Figure 8: Data Efficiency for different benchmarks for LaMO with Transolver (Top) (a) Navier Stokes, (b) Darcy (c) Airfoil, (Buttom) (d) Pipe, (e) Plasticity and (f) Elasticity benchmark respectively [PITH_FULL_IMAGE:figures/full_fig_p027_8.png]

**Figure 9.** Figure 9: Hidden Attention Matrices comparison of Transolver (Top) with LaMO (Bottom) across the layers on Airfoil benchmark. Layer 1 Layer 2 Layer 3 Layer 4 Layer 5 Layer 6 Layer 7 Layer 8 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 Transolver Ours [PITH_FULL_IMAGE:figures/ful…

**Figure 10.** Figure 10: Hidden Attention Matrices comparison of Transolver (Top) with LaMO (Bottom) across the layers on Elasticity benchmark. 1 2 3 4 5 6 7 8 0 0.2 0.4 0.6 0.8 1 Layer Index Cosine Similarity (a) Airfoil 1 2 3 4 5 6 7 8 0 0.2 0.4 0.6 0.8 1 Layer Index Cosine Similarity (b) E…

**Figure 11.** Figure 11: Intercosine similarity of tokens across layers for LaMO and Transolver on (a) Airfoil and (b) Elasticity benchmark, respectively. G. Limitations and Future Work LaMO demonstrates promising results in solving parametric PDEs, but its efficiency in an unsupervised setti…

**Figure 12.** Figure 12: Model Prediction Comparison: The figure compares Transolver and LaMO on the Darcy dataset. The (Top) row represents the ground truth, the (Middle) row shows the predictions from Transolver and LaMO, and the (Bottom) row illustrates the error heatmap, capturing the dif…

**Figure 13.** Figure 13: Model Prediction Comparison: The figure compares Transolver and LaMO on the Navier Stokes dataset. The (Top) row represents the ground truth, the (Middle) row shows the predictions from Transolver and LaMO, and the (Bottom) row illustrates the error heatmap, capturing…

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

SPAMoE: Spectrum-Aware Hybrid Operator Framework for Full-Waveform Inversion
cs.LG 2026-04 unverdicted novelty 7.0 of 10

SPAMoE reduces average MAE by 44.4% on ten OpenFWI sub-datasets via a spectral-preserving DINO encoder plus frequency-routed MoE of FNO, MNO and LNO experts.
Structure-Preserving Learning Improves Geometry Generalization in Neural PDEs
cs.LG 2026-02 conditional novelty 6.0 of 10

A geometry-conditioned Whitney-form neural network that solves a learned discrete conservation law improves out-of-distribution geometry generalization for steady-state PDEs compared with regression-based neural operators.
Data-free neural PDE solvers based on Graph Neural Networks and weak forms
cs.CE 2026-07 conditional novelty 5.0 of 10

A graph-neural-network PDE solver trained on the weak-form force residual — no simulation data — reports residual convergence below 1% on unseen load cases and one modified geometry, with residual-based test-time refinement.

Reference graph

Works this paper leans on

34 extracted references · 9 canonical work pages · cited by 3 Pith papers

[1]

An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv: 2010.11929,

Alexey, D. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv: 2010.11929,

arXiv 2010
[6]

Training configurations are directly from previous works without extra tuning

Training and model configurations of LaMO. Training configurations are directly from previous works without extra tuning. For the Darcy dataset, we adopt an additional spatial gradient regularization terml gdl following ONO work. CONFIGURATION BENCHMARKS DARCYNAVIER–STOKESELASTICITYPLASTICITYAIRFOILPIPE TRAINING LOSSFUNCTIONl 2 + 0.1lgdl RELATIVEl2 EPOCHS...

work page 2023
[9]

Efficiently modeling long sequences with structured state spaces.arXiv preprint arXiv:2111.00396, 2021a

Gu, A., Goel, K., and R ´e, C. Efficiently modeling long sequences with structured state spaces.arXiv preprint arXiv:2111.00396, 2021a. Gu, A., Johnson, I., Goel, K., Saab, K., Dao, T., Rudra, A., and R´e, C. Combining recurrent, convolutional, and continuous-time models with linear state space layers. Advances in neural information processing systems, 34...

arXiv
[11]

Dpot: Auto-regressive denoising operator transformer for large-scale pde pre- training.arXiv preprint arXiv:2403.03542,

Hao, Z., Su, C., Liu, S., Berner, J., Ying, C., Su, H., Anand- kumar, A., Song, J., and Zhu, J. Dpot: Auto-regressive denoising operator transformer for large-scale pde pre- training.arXiv preprint arXiv:2403.03542,

arXiv
[14]

Gateloop: Fully data-controlled linear recurrence for sequence modeling.arXiv preprint arXiv:2311.01927,

Katsch, T. Gateloop: Fully data-controlled linear recurrence for sequence modeling.arXiv preprint arXiv:2311.01927,

arXiv
[15]

Neural operator: Learning maps between function spaces.arXiv preprint arXiv:2108.08481,

Kovachki, N., Li, Z., Liu, B., Azizzadenesheli, K., Bhat- tacharya, K., Stuart, A., and Anandkumar, A. Neural operator: Learning maps between function spaces.arXiv preprint arXiv:2108.08481,

arXiv
[16]

Fourier neural operator for parametric partial differential equa- tions.arXiv preprint arXiv:2010.08895,

Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhat- tacharya, K., Stuart, A., and Anandkumar, A. Fourier neural operator for parametric partial differential equa- tions.arXiv preprint arXiv:2010.08895,

arXiv 2010
[17]

Z., Liu, B., and Anandkumar, A

Li, Z., Huang, D. Z., Liu, B., and Anandkumar, A. Fourier neural operator with learned deformations for pdes on general geometries.arXiv preprint arXiv:2207.05209, 2022a. Li, Z., Meidani, K., and Farimani, A. B. Transformer for partial differential equations’ operator learning.arXiv preprint arXiv:2205.13671, 2022b. Li, Z., Shu, D., and Barati Farimani, A...

arXiv

Show all 34 references

[18]

Ht-net: Hierarchical trans- former based operator learning model for multiscale pdes

Liu, X., Xu, B., and Zhang, L. Ht-net: Hierarchical trans- former based operator learning model for multiscale pdes. arXiv preprint arXiv:2210.10890,

arXiv
[19]

Liu, Z., Lin, Y ., Cao, Y ., Hu, H., Wei, Y ., Zhang, Z., Lin, S., and Guo, B

URLhttps://arxiv.org/abs/2401.10166. Liu, Z., Lin, Y ., Cao, Y ., Hu, H., Wei, Y ., Zhang, Z., Lin, S., and Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. InProceedings of the IEEE/CVF international conference on computer vision, pp. 10012–10022,

arXiv
[20]

Fixing weight decay regu- larization in adam.arXiv preprint arXiv:1711.05101, 5,

Loshchilov, I., Hutter, F., et al. Fixing weight decay regu- larization in adam.arXiv preprint arXiv:1711.05101, 5,

arXiv
[21]

A survey of mamba.arXiv preprint arXiv:2408.01129,

Qu, H., Ning, L., An, R., Fan, W., Derr, T., Liu, H., Xu, X., and Li, Q. A survey of mamba.arXiv preprint arXiv:2408.01129,

arXiv
[22]

A., Ross, Z

Rahman, M. A., Ross, Z. E., and Azizzadenesheli, K. U-no: U-shaped neural operators.arXiv preprint arXiv:2204.11127,

arXiv
[25]

Tran, A., Mathews, A., Xie, L., and Ong, C. S. Factorized fourier neural operators.arXiv preprint arXiv:2111.13802,

arXiv
[28]

Tran- solver: A fast transformer solver for pdes on general geometries.arXiv preprint arXiv:2402.02366,

Wu, H., Luo, H., Wang, H., Wang, J., and Long, M. Tran- solver: A fast transformer solver for pdes on general geometries.arXiv preprint arXiv:2402.02366,

arXiv
[29]

Improved operator learning by orthogonal attention.arXiv preprint arXiv:2310.12487,

Xiao, Z., Hao, Z., Lin, B., Deng, Z., and Su, H. Improved operator learning by orthogonal attention.arXiv preprint arXiv:2310.12487,

arXiv
[30]

Vision mamba: Efficient visual representation learn- ing with bidirectional state space model.arXiv preprint arXiv:2401.09417,

Zhu, L., Liao, B., Zhang, Q., Wang, X., Liu, W., and Wang, X. Vision mamba: Efficient visual representation learn- ing with bidirectional state space model.arXiv preprint arXiv:2401.09417,

arXiv
[31]

Proof:Let xi ∈Ω denote the i-th element in the input domain, and let zj represent the j-th latent token in the latent domain Ωs

IfΩis a countable domain, the latent domainΩ s is isomorphic toΩ. Proof:Let xi ∈Ω denote the i-th element in the input domain, and let zj represent the j-th latent token in the latent domain Ωs. The latent weight of xi with respect to zj is denoted by wxi,zj ∈R . Given a const...

2024
[32]

In the experiment, 900 samples with varying die shapes are used for training, while 80 additional samples are reserved for testing

The output is the deformation of each mesh point over 20 future timesteps, recorded as a tensor of shape 20×101×31×4 , capturing deformation in four directions. In the experiment, 900 samples with varying die shapes are used for training, while 80 additional samples are reserv...

2020
[34]

LaMO significantly reduces inter-cosine similarity across layers compared to Transolver, indicating better representation learning through SSM. This improvement can be attributed to the absence of the softmax operation, which in Transolver has been identified as causing over-s...

2023
[1990]

A., Shen, Q., Kawaguchi, K., and Karniadakis, G

Hu, Z., Daryakenari, N. A., Shen, Q., Kawaguchi, K., and Karniadakis, G. E. State-space models are accurate and efficient neural operators for dynamical systems.arXiv preprint arXiv:2409.03231,

arXiv
[2001]

and Dao, T

Gu, A. and Dao, T. Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752,

arXiv
[2004]

and Gu, A

Dao, T. and Gu, A. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality.arXiv preprint arXiv:2405.21060,

arXiv
[2005]

Cono: Complex neural operator for continous dynamical physical systems

Tiwari, K., Krishnan, N., and Prathosh, A. Cono: Complex neural operator for continous dynamical physical systems. arXiv preprint arXiv:2406.02597,

arXiv
[2007]

Solving high-dimensional pdes with latent spectral models.arXiv preprint arXiv:2301.12664,

Wu, H., Hu, T., Luo, H., Wang, J., and Long, M. Solving high-dimensional pdes with latent spectral models.arXiv preprint arXiv:2301.12664,

arXiv
[2016]

Poseidon: Efficient foundation models for pdes.arXiv preprint arXiv:2405.19101,

Herde, M., Raoni ´c, B., Rohner, T., K ¨appeli, R., Moli- naro, R., de B ´ezenac, E., and Mishra, S. Poseidon: Efficient foundation models for pdes.arXiv preprint arXiv:2405.19101,

arXiv
[2017]

Anti- oversmoothing in deep vision transformers via the fourier domain analysis: From theory to practice.arXiv preprint arXiv:2203.05962,

Wang, P., Zheng, W., Chen, T., and Wang, Z. Anti- oversmoothing in deep vision transformers via the fourier domain analysis: From theory to practice.arXiv preprint arXiv:2203.05962,

arXiv
[2018]

H., Senocak, A., Feng, J., and Chung, J

Erol, M. H., Senocak, A., Feng, J., and Chung, J. S. Audio mamba: Bidirectional state space model for audio rep- resentation learning.arXiv preprint arXiv:2406.03344,

arXiv
[2020]

Centered self-attention layers.arXiv preprint arXiv:2306.01610,

Ali, A., Galanti, T., and Wolf, L. Centered self-attention layers.arXiv preprint arXiv:2306.01610,

arXiv
[2021]

Adaptive fourier neural operators: Efficient token mixers for transformers.arXiv preprint arXiv:2111.13587,

Guibas, J., Mardani, M., Li, Z., Tao, A., Anandkumar, A., and Catanzaro, B. Adaptive fourier neural operators: Efficient token mixers for transformers.arXiv preprint arXiv:2111.13587,

arXiv
[2022]

U-net: Con- volutional networks for biomedical image segmentation

Ronneberger, O., Fischer, P., and Brox, T. U-net: Con- volutional networks for biomedical image segmentation. InMedical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Confer- ence, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18...

2015
[2023]

A., Sieber, J., and Zeilinger, M

Alonso, C. A., Sieber, J., and Zeilinger, M. N. State space models as foundation models: A control theoretic overview.arXiv preprint arXiv:2403.16899,

arXiv
[2024]

and Oseledets, I

Fanaskov, V . and Oseledets, I. Spectral neural operators. arXiv preprint arXiv:2205.10573,

arXiv
[2025]

Bevanda, P., Sosnowski, S., and Hirche, S

URL https: //arxiv.org/abs/2501.06007. Bevanda, P., Sosnowski, S., and Hirche, S. Koopman oper- ator dynamical models: Learning, analysis and control. Annual Reviews in Control, 52:197–212,

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv: 2010.11929,

Alexey, D. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv: 2010.11929,

arXiv 2010

[2] [6]

Training configurations are directly from previous works without extra tuning

Training and model configurations of LaMO. Training configurations are directly from previous works without extra tuning. For the Darcy dataset, we adopt an additional spatial gradient regularization terml gdl following ONO work. CONFIGURATION BENCHMARKS DARCYNAVIER–STOKESELASTICITYPLASTICITYAIRFOILPIPE TRAINING LOSSFUNCTIONl 2 + 0.1lgdl RELATIVEl2 EPOCHS...

work page 2023

[3] [9]

Efficiently modeling long sequences with structured state spaces.arXiv preprint arXiv:2111.00396, 2021a

Gu, A., Goel, K., and R ´e, C. Efficiently modeling long sequences with structured state spaces.arXiv preprint arXiv:2111.00396, 2021a. Gu, A., Johnson, I., Goel, K., Saab, K., Dao, T., Rudra, A., and R´e, C. Combining recurrent, convolutional, and continuous-time models with linear state space layers. Advances in neural information processing systems, 34...

arXiv

[4] [11]

Dpot: Auto-regressive denoising operator transformer for large-scale pde pre- training.arXiv preprint arXiv:2403.03542,

Hao, Z., Su, C., Liu, S., Berner, J., Ying, C., Su, H., Anand- kumar, A., Song, J., and Zhu, J. Dpot: Auto-regressive denoising operator transformer for large-scale pde pre- training.arXiv preprint arXiv:2403.03542,

arXiv

[5] [14]

Gateloop: Fully data-controlled linear recurrence for sequence modeling.arXiv preprint arXiv:2311.01927,

Katsch, T. Gateloop: Fully data-controlled linear recurrence for sequence modeling.arXiv preprint arXiv:2311.01927,

arXiv

[6] [15]

Neural operator: Learning maps between function spaces.arXiv preprint arXiv:2108.08481,

Kovachki, N., Li, Z., Liu, B., Azizzadenesheli, K., Bhat- tacharya, K., Stuart, A., and Anandkumar, A. Neural operator: Learning maps between function spaces.arXiv preprint arXiv:2108.08481,

arXiv

[7] [16]

Fourier neural operator for parametric partial differential equa- tions.arXiv preprint arXiv:2010.08895,

Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhat- tacharya, K., Stuart, A., and Anandkumar, A. Fourier neural operator for parametric partial differential equa- tions.arXiv preprint arXiv:2010.08895,

arXiv 2010

[8] [17]

Z., Liu, B., and Anandkumar, A

Li, Z., Huang, D. Z., Liu, B., and Anandkumar, A. Fourier neural operator with learned deformations for pdes on general geometries.arXiv preprint arXiv:2207.05209, 2022a. Li, Z., Meidani, K., and Farimani, A. B. Transformer for partial differential equations’ operator learning.arXiv preprint arXiv:2205.13671, 2022b. Li, Z., Shu, D., and Barati Farimani, A...

arXiv