Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Symbolic Regression with Multimodal Large Language Models and Kolmogorov Arnold Networks

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A vision-capable LLM shown only a plot can propose ansätze that evolve into exact closed forms, and a Kolmogorov–Arnold decomposition lifts the trick to multivariate functions.

desk verdict A genuinely new vision-LLM + FunSearch + KAN pipeline for symbolic regression; the univariate part is solid and honestly scoped, the multivariate extension is a proof of concept resting on an unproven simplicity assumption. read the letter →

arxiv 2505.07956 v1 pith:OEO52C5Z submitted 2025-05-12 cs.LG cs.NEcs.SC

Thomas R. Harvey , Fabian Ruehle , Kit Fraser-Taliente , James Halverson This is my paper · ORCID
classification cs.LGcs.NEcs.SC
keywords symbolicregressionmultimodallargelanguagemodelsKolmogorov-Arnoldnetworksgeneticalgorithmvisualfunctiondiscoverypromptengineeringspecialfunctionsexactexpressionrecovery
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that symbolic regression can be run by showing a vision-capable large language model a plot of a function and asking it to write an ansatz, then fitting the free parameters and evolving the ansätze with a genetic algorithm. The authors argue this works because the LLM's pretraining instils a human-like simplicity bias: it proposes compact functional forms without being given any basis-function library. They then extend the same idea to multivariate targets through Kolmogorov–Arnold Networks, fitting each univariate edge function separately, composing the results, and compressing them with symbolic simplification and a second LLM pass. If the approach holds, interpretable expressions can be recovered from pure visual input, with prompt conditioning as a cheap way to steer the search.

What carries the argument

The machinery has two parts. First, a genetic algorithm over LLM proposals: a population of lambda functions is scored by a scale-normalised residual that emphasises shape rather than large values; pairs are sampled by softmax over scores and handed to the LLM as two example candidates plus the plot, and the reply is the next generation's ansatz. Second, a Kolmogorov–Arnold Network—a layered graph whose edges carry univariate spline functions instead of weighted sums—provides the decomposition: pruning the trained KAN reduces the architecture, and each surviving edge becomes a univariate symbolic-regression problem. The final pass compounds symbolic simplification, coefficient refitting, and an LLM-guided step that drops negligible terms and recognises Taylor-series patterns, which matters because per-edge constants are not unique and composition does not commute with fitting.

What would settle it

Train a KAN on dense samples of $f(x,y)=\sqrt{|x-y|}$, prune it, and run KAN-LEx on each edge for twenty independent seeds with the default prompts. If the composed expression recovers the target at the paper's default score threshold of $10^{-5}$ rarely or not at all, the claim that edge simplicity is not an obstacle in practice fails for functions whose Kolmogorov–Arnold decomposition forces a cusped univariate edge.

Watch

Extended reading notes

Core claim

The paper claims that symbolic regression can be decomposed into univariate regressions and solved by a vision-capable LLM: shown a plot of a univariate function, the model proposes a compact Python-$\lambda$ ansatz; numerical fitting fixes its parameters; and a genetic loop, seeded with two scored candidates and prompted to give an improved ansatz, evolves a population until exact recovery. On a 20-function benchmark this loop recovers exact expressions more often than the proprietary symbolic-search baseline. The multivariate extension, KAN-LEx, is the sharper claim: after training and pruning a Kolmogorov–Arnold Network, the target function is a sum of composed univariate edge functions, and applying the same plot-driven fitting edge by edge yields, after symbolic simplification and refitting, the exact closed form in all three worked examples ($\exp[\sin(\pi x)+y^2]$, $xy$, and $xy/z$).

Load-bearing premise

The load-bearing premise is that a pruned KAN's edge functions are simple enough that a vision LLM can guess their closed forms from plots; the paper admits there is no guarantee of this, only that its examples suggest it.

Editorial extensions

If this is right

  • On a 20-function benchmark, the plot-only loop recovers the exact expression in 12 cases, while the proprietary symbolic-search baseline recovers 8; exact recovery is therefore not an occasional accident.
  • Because no basis-function library is specified, the search can be conditioned by prompt wording alone: one prompt steers the model to return an exact Bessel-function call rather than a trigonometric approximation.
  • For multivariate targets, pruning a KAN narrows the architecture before per-edge fitting, and the worked examples—an exponential of sine-plus-square, a product, and Newton's gravitational law—all reduce to the correct closed form after composition and simplification.
  • The method tolerates modest noise and still finds the ground-truth trend, but the user must raise the exit threshold to match the noise level, otherwise the loop overfits.
  • Open-weight local models can match the proprietary model's exact-recovery count but take substantially longer, so the approach does not depend on one commercial API.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Our inference: the boundary of the method is the model's pretraining vocabulary; fine-tuning a small vision-language model on synthetic plots of specialised functions should extend exact recovery to domains where a general model only approximates.
  • Our inference: plotting is an information bottleneck—features below plot resolution are invisible to the LLM, so multi-resolution crops or zoomed views of the same function would be a cheap test of whether the vision input is the limiting step.
  • Our inference: the same loop could be aimed at physics-law discovery by prompting for expected symmetries, units, or leading terms; the paper lists domain prompting as future work, but treating it as a tunable prior is a natural next experiment.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces LLM-LEx, a symbolic regression method in which a vision-capable LLM is prompted with a plot of a univariate function and two parent candidate expressions and asked to propose a Python lambda ansatz; free parameters are fitted numerically and the population is evolved with a genetic algorithm inspired by FunSearch. The authors benchmark LLM-LEx against Mathematica's FindFormula on 20 generated functions, test open-source LLMs, study robustness to noise, and show that special functions can be recovered through prompt engineering. They then propose KAN-LEx, which trains and prunes a Kolmogorov-Arnold network, fits each edge function with LLM-LEx, composes the resulting univariate expressions, and simplifies the result with sympy and an LLM. Three multivariate examples are reported: exp[sin(pi x)+y^2], xy, and xy/z. The paper includes a public GitHub repository and explicitly disclaims state-of-the-art performance.

Significance. The central idea—using an LLM's visual pattern recognition as a symbolic-ansatz proposer inside an evolutionary loop—is novel and well-motivated, and the authors are appropriately modest about positioning the work as a proof of concept rather than a competitive benchmark. The availability of code and notebooks is a strength, as is the demonstration that open-source models can run the method locally. If the results are reproducible, the paper makes a useful contribution to the growing literature on LLM-guided symbolic regression. However, the multivariate claim that 'univariate is all you need' is currently under-supported: it rests on an empirical assumption about KAN edge simplicity that the paper itself concedes is unproven, and the three multivariate examples are all favorable cases. The univariate evaluation is also thin, with no comparison to standard genetic-programming baselines such as PySR or Operon. These issues, together with a concrete error in the scoring formula, mean that the paper's significance depends on the requested revisions.

major comments (3)
  1. [§3.1 (footnote 3), §3.2] The multivariate extension rests on the assumption that each edge function of a trained, pruned KAN is simple enough for the plot-based LLM to propose a correct symbolic ansatz. The paper itself concedes in footnote 3 that 'there is no theoretical guarantee the required univariate functions are simple or remotely well behaved,' and the three examples in §3.2—exp[sin(pi x)+y^2], xy, and xy/z—are all low-dimensional and all succeed. This is not sufficient to support the abstract's claim that 'univariate is all you need' for symbolic regression, especially since Table 3 reports only 12/20 exact recoveries for gpt-4o on univariate functions; a KAN edge from a general target could be no simpler. I ask for a systematic multivariate evaluation with reported success/failure rates on a broader set of targets, including cases where the Kolmogorov-Arnold decomposition is not visually simple, or, alternatively, a substantial rewording of the multivariate claim to state that the method applies when the trained KAN edge functions are simple.
  2. [§2.2, Eq. (4)] The definition of MADy as median({yi - median({yj})}) is mathematically incorrect: the median of the centered values yi - median(y) is identically zero for any finite sample. Consequently, the max in global-scale is vacuous and global-scale always reduces to alpha * mean(|yi|), contrary to the stated intention of a 'non-vanishing characteristic scale.' The presumably intended quantity is the median absolute deviation, median(|yi - median(y)|). Since the score in Eq. (3) drives selection and termination in every reported experiment, this error is load-bearing; the definition should be corrected and the authors should state whether any reported scores or thresholds change as a result.
  3. [§2.3, Tables 1 and 3] The central claim that LLM-LEx is 'viable and surprisingly successful' is supported only by a comparison to Mathematica's FindFormula on 20 functions, with one or two stochastic LLM runs per function and manual inspection of correctness. The paper cites PySR [1] and Operon [2] but does not compare against them, so it is unclear how the method fares relative to standard symbolic regression packages on the same benchmark. Because the LLM responses are probabilistic, the reported 'Total Correct' counts in Table 3 need to be accompanied by repeated-run success rates (e.g., success probability over at least several runs per function) and ideally confidence intervals. This additional evidence is necessary to substantiate the univariate viability claim.
minor comments (5)
  1. [§2, first paragraph] Typo: 'throuhout' should be 'throughout'; in §2.3, 'at the expensive of some quality' should be 'at the expense of some quality.'
  2. [§2.2, Eq. (4)] The notation 'mean({|yi|}, ϵ)' is undefined; please specify how the small hyperparameter ϵ enters the mean (for example, as a floor or an additive offset).
  3. [Table 3 caption] The caption states 'All runs with LLM-LEx were single shot,' which appears to conflict with Table 1's '# LLM-LEx runs' column reporting values of 1 and 2; please clarify the intended meaning of 'single shot' and reconcile the two tables.
  4. [§3.2, Example 2, Eq. (17)] The list of returned expressions is missing a comma between 'y*x' and 'x**1 * y**1', making the list syntactically confusing.
  5. [Figure 5] The caption says that below the dashed red line the genetic algorithm is stopped early, but the score threshold corresponding to the line is not stated; please add this value.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: KAN-LEx's outputs are fits to external ground-truth functions; self-citations to the KAN paper are implementation support, not load-bearing premises.

full rationale

The paper's derivation chain is not circular. In LLM-LEx, the LLM proposes a symbolic ansatz from a plot image, the free parameters are fitted by scipy optimizers, and the algorithm is benchmarked against externally generated ground-truth expressions (Tables 1 and 2). Nothing in the scoring metric (Eq. 3), the softmax selection (Eq. 5), or the genetic algorithm encodes the target expression; the target appears only as data and as the external benchmark. In KAN-LEx, the Kolmogorov-Arnold theorem is a standard mathematical fact, and the KAN paper [16] is cited for the concrete network architecture and pruning implementation. Although [16] shares authors with the present paper, it is not invoked as a uniqueness theorem and does not forbid alternatives; the paper explicitly states that 'in principle any symbolic regression method can be used for this step.' The multivariate examples (Sections 3.2) use external target expressions such as exp[sin(pi x) + y^2], xy, and xy/z, and the recovered forms are fits to the same data the KAN was trained on; this is the stated goal of symbolic regression, not a prediction forced by the method's definitions. Footnote 3 concedes that 'there is no theoretical guarantee the required univariate functions are simple or remotely well behaved'; this is an honest robustness limitation, not a circular step. The only self-referential element is the citation to the authors' own KAN implementation, but it is implementation support rather than a load-bearing premise, so it does not raise the circularity score.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The ledger contains no fitted constants because the paper does not derive a law; the recovered coefficients are outputs of the search. The listed hyperparameters are hand-chosen settings of the scoring and selection procedure. The most influential assumptions are the LLM's pretraining prior and the simplicity of KAN edge functions, both of which the authors acknowledge only partially.

free parameters (4)
  • alpha = 0.01
    Bracketing constant in the score function (Eq. 3) that balances absolute and relative residuals; chosen by hand, not fitted.
  • epsilon = small, not specified numerically
    Introduced in Eq. 4 to avoid division by zero in global-scale; the exact value is not reported.
  • selection temperature T = 1
    Softmax temperature (Eq. 5) controlling genetic selection pressure; set to 1 unless stated otherwise.
  • population size and generation count = 25, 10
    Fixed for all LLM-LEx benchmark runs (Section 2.3); results would vary with these search settings.
assumptions (4)
  • standard math Kolmogorov-Arnold representation theorem
    Invoked in Section 3.1 to justify representing any multivariate function as sums of univariate functions; this is a known theorem with mild regularity assumptions.
  • domain assumption LLM pretraining gives a useful prior for functional forms
    The method assumes gpt-4o or similar models have seen enough plotted functions during pretraining to produce sensible ansatze from an image alone (Section 2.1).
  • domain assumption KAN edge functions are simple enough to be fit by a vision LLM
    Section 3.1 footnote admits there is no theoretical guarantee the required univariate functions are simple; the successful examples assume the pruned KAN edges are visually identifiable.
  • domain assumption Sampled data and plots reveal the true functional form
    The benchmark uses 100 evenly spaced points over [0.1,5] (Section 2.3); the cos(e^x) alias discussion in Section 2.4 shows undersampling can mislead the model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Symbolic Regression with Multimodal Large Language Models and Kolmogorov Arnold Networks." pith.science (2026). https://pith.science/paper/OEO52C5Z

@misc{pith2026250507956,
  author       = {Pith},
  title        = {Pith review of: Symbolic Regression with Multimodal Large Language Models and Kolmogorov Arnold Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OEO52C5Z}},
  note         = {Machine review of arXiv:2505.07956}
}
read the original abstract

We present a novel approach to symbolic regression using vision-capable large language models (LLMs) and the ideas behind Google DeepMind's Funsearch. The LLM is given a plot of a univariate function and tasked with proposing an ansatz for that function. The free parameters of the ansatz are fitted using standard numerical optimisers, and a collection of such ans\"atze make up the population of a genetic algorithm. Unlike other symbolic regression techniques, our method does not require the specification of a set of functions to be used in regression, but with appropriate prompt engineering, we can arbitrarily condition the generative step. By using Kolmogorov Arnold Networks (KANs), we demonstrate that ``univariate is all you need'' for symbolic regression, and extend this method to multivariate functions by learning the univariate function on each edge of a trained KAN. The combined expression is then simplified by further processing with a language model.

Figures

Figures reproduced from arXiv: 2505.07956 by the authors.

Figure 1
Figure 1. An example function to learn 2.1 A simple example Consider the graph in [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The structure of LLM-LEx. model has been previously explored in the context of Funsearch [4, 8, 9]. We suggest integrating this approach into the framework introduced in the previous section. The general structure of our approach is indicated in [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Four example functions found by LLM-LEx with various scores. The func [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Dense vs. Sparse sampling of x values for f(x) = 4.67315 + cos(exp(x)). out of four runs, yielding a 4/5 total success rate. Given the probabilistic nature of language models at non-zero temperature, this variability is unsurprising. 2.5 Adding noise Thus far, LLM-LEx …
Figure 5
Figure 5. Figure 5: Compute times and scores of LLM-LEx across different LLMs. Below the [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: The general structure of KAN-LEx. The visual symbolic regression technique we have introduced is developed for univari￾ate functions f : R → R and utilises 2D images. However, it is not applicable to multivariate functions, since these cannot be plotted in 2D. Luckily,…
Figure 7
Figure 7. Figure 7: Selection stages for the KAN architecture. The final model is a [2 [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Selection stages for the KAN architecture. The final model is a [2 [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DeepInflation: an AI agent for research and model discovery of inflation

    astro-ph.CO 2026-01 conditional novelty 6.0 of 10

    An LLM agent with symbolic regression finds simple inflation potentials that match target CMB observables, but the outputs are fitted to the targets rather than independently predicted.

Reference graph

Works this paper leans on

17 extracted references · 6 canonical work pages · cited by 1 Pith paper

  1. [1]

    Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl,

    M. Cranmer, “Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl,”arXiv e-prints(May, 2023) arXiv:2305.01582, arXiv:2305.01582 [astro-ph.IM]

    arXiv 2023

  2. [2]

    Operon c++: An efficient genetic programming framework for symbolic regression,

    B. Burlacu, G. Kronberger, and M. Kommenda, “Operon c++: An efficient genetic programming framework for symbolic regression,” inProceedings of the 2020 Genetic and Evolutionary Computation Conference Companion, GECCO ’20, p. 1562–1570. Association for Computing Machinery, New York, NY, USA, 2020.https://doi.org/10.1145/3377929.3398099

    arXiv 2020

  3. [3]

    Exhaustive Symbolic Regression,

    D. J. Bartlett, H. Desmond, and P. G. Ferreira, “Exhaustive Symbolic Regression,”IEEE Trans. Evol. Comput.28no. 4, (2024) 964, arXiv:2211.11461 [astro-ph.CO]

    arXiv 2024

  4. [4]

    Mathematical discoveries from program search with large language models,

    B. Romera-Paredes, M. Barekatain, A. Novikov, M. Balog, M. P. Kumar, E. Dupont, F. J. Ruiz, J. S. Ellenberg, P. Wang, O. Fawzi,et al., “Mathematical discoveries from program search with large language models,”Nature625 no. 7995, (2024) 468–475

    work page 2024

  5. [5]

    Llm-lex

    “Llm-lex.”https://github.com/harveyThomas4692/llmlex

    work page

  6. [6]

    Mathematica, Version 14.2

    W. R. Inc., “Mathematica, Version 14.2.” https://www.wolfram.com/mathematica. Champaign, IL, 2024

    2024

  7. [7]

    A survey of parallel genetic algorithms,

    E. Cant´ u-Pazet al., “A survey of parallel genetic algorithms,”Calculateurs paralleles, reseaux et systems repartis10no. 2, (1998) 141–171

    work page 1998

  8. [8]

    Refining Integration-by-Parts Reduction of Feynman Integrals with Machine Learning,

    M. von Hippel and M. Wilhelm, “Refining Integration-by-Parts Reduction of Feynman Integrals with Machine Learning,”arXiv:2502.05121 [hep-th]

    arXiv

Show all 17 references
  1. [9]

    Generative modeling for mathematical discovery,

    J. S. Ellenberg, K. Fraser-Taliente, T. R. Harvey, K. Srivastava, and A. V. Sutherland, “Generative modeling for mathematical discovery,” 2025. https://arxiv.org/abs/2503.11061

    2025 arXiv

  2. [10]

    Contemporary symbolic regression methods and their relative performance,

    W. G. L. Cava, P. Orzechowski, B. Burlacu, F. O. de Fran¸ ca, M. Virgolin, Y. Jin, M. Kommenda, and J. H. Moore, “Contemporary symbolic regression methods and their relative performance,”CoRRabs/2107.14351(2021) ,2107.14351

    2021 arXiv

  3. [11]

    R. Poli, W. B. Langdon, N. F. McPhee, and J. R. Koza,A Field Guide to Genetic Programming. lulu.com, 2008. 24

    2008

  4. [12]

    Llm-sr: Scientific equation discovery via programming with large language models,

    P. Shojaee, K. Meidani, S. Gupta, A. B. Farimani, and C. K. Reddy, “Llm-sr: Scientific equation discovery via programming with large language models,” arXiv preprint arXiv:2404.18400(2024)

    2024 arXiv

  5. [13]

    In-context symbolic regression: Leveraging large language models for function discovery,

    M. Merler, K. Haitsiukevich, N. Dainese, and P. Marttinen, “In-context symbolic regression: Leveraging large language models for function discovery,”arXiv preprint arXiv:2404.19094(2024)

    2024 arXiv

  6. [14]

    Symbolic regression with a learned concept library,

    A. Grayeli, A. Sehgal, O. Costilla Reyes, M. Cranmer, and S. Chaudhuri, “Symbolic regression with a learned concept library,”Advances in Neural Information Processing Systems37(2024) 44678–44709

    2024

  7. [15]

    Visymre: Vision-guided multimodal symbolic regression,

    D. Li, J. Yin, J. Xu, X. Li, and J. Zhang, “Visymre: Vision-guided multimodal symbolic regression,”arXiv preprint arXiv:2412.11139(2024)

    2024

  8. [16]

    KAN: Kolmogorov-Arnold Networks,

    Z. Liu, Y. Wang, S. Vaidya, F. Ruehle, J. Halverson, M. Soljaˇ ci´ c, T. Y. Hou, and M. Tegmark, “KAN: Kolmogorov-Arnold Networks,”arXiv:2404.19756 [cs.LG]

    arXiv

  9. [17]

    Sympy: symbolic computing in python,

    A. Meurer, C. P. Smith, M. Paprocki, O. ˇCert´ ık, S. B. Kirpichev, M. Rocklin, A. Kumar, S. Ivanov, J. K. Moore, S. Singh, T. Rathnayake, S. Vig, B. E. Granger, R. P. Muller, F. Bonazzi, H. Gupta, S. Vats, F. Johansson, F. Pedregosa, M. J. Curry, A. R. Terrel, v. Rouˇ cka, A....

    2017

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.