Pith. sign in

REVIEW 4 major objections 6 minor 22 references

Learning Semantics-aware Search Operators for Genetic Programming

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a graph neural network can learn to score one-step program expansions by their prospective usefulness, and that grafting the top-scoring fragments into an evolving population improves symbolic regression success and…

desk verdict A solid, clearly written neuro-guided GP paper whose central claim is undercut by single-run experiments with no significance tests; worth reviewing, but needs repeated runs. read the letter →

arxiv 2502.04568 v1 pith:KBRR7UG6 submitted 2025-02-06 cs.LG cs.NE

Piotr Wyrwiński , Krzysztof Krawiec This is my paper · ORCID
classification cs.LGcs.NE
keywords geneticprogrammingsymbolicregressiongraphneuralnetworkssemantics-awaresearchoperatorssaliencymapprogramsynthesissubprogramlibraryneuro-guidedevolution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a genetic programming (GP) search for symbolic regression—finding a closed-form expression that fits data—can be guided by a learned, semantics-aware operator rather than by fitness alone. The proposed operator, NEON, uses a graph neural network to score one-step expansions of program subtrees by their prospective usefulness, then grafts the top-scoring fragments into evolving programs through a library. On a suite of 97 physics-derived regression equations, NEON improves the success rate over conventional tree-based GP at populations 100, 200, and 500, and the solutions it finds are several times smaller. The learned guidance also beats a random-selection ablation at those population sizes, with the ablation slightly ahead only at the largest tested population.

What carries the argument

The central object is the expanded graph representation paired with the GNN saliency mechanism. Application nodes represent DSL operations applied to concrete arguments, and value nodes store the vector of outputs an expression produces across the training data, so the graph exposes program semantics to the network rather than just syntax. The GNN follows the graph attention architecture, runs several rounds of message passing, and outputs a saliency score in (0,1) for each candidate expansion; the paper keeps nodes scoring above 0.5, takes the top five, and enqueues them into a FIFO library whose capacity equals the population size. The same trained network is applied to every test problem, so all learning happens in this saliency predictor before the evolutionary runs.

What would settle it

Retrain the GNN with the same inputs but with the saliency labels shuffled, rerun the 97-problem suite, and check whether success rates remain at NEON levels; if they do, the learned saliency map is not the cause of the improvement.

Watch

Extended reading notes

Core claim

NEON replaces the conventional crossover/mutation pipeline with a grafting operator that inserts library programs into selected parents. To fill the library, the method samples parent programs, extracts subtrees, expands each subtree by one level in every way the instruction set allows, and builds a graph whose value nodes carry the actual outputs those expressions produce over all training examples. A GNN, trained once on synthetic expression trees to label an expanded node as positive exactly when that node occurs in the upper part of a target tree, converts this graph into a saliency map over candidate expansions; the top-scoring candidates enter the library, and grafting replaces a random subtree of a parent with a uniformly drawn library program. The paper's working hypothesis is that this extra prospective-value guidance improves the success rate of otherwise rudimentary GP, and the reported experiments support that claim on most settings while showing much more compact solutions.

Load-bearing premise

The method stands on the assumption that a node appearing in the upper part of a randomly sampled synthetic target tree is a reliable signal that the same expansion will be prospectively useful when the GNN is applied to real physics-derived regression problems.

Editorial extensions

If this is right

  • At population size 100, NEON's success rate is 13.4% versus 7.2% for baseline GP; at population 500 it is 19.6% versus 17.5%.
  • Average sizes of successfully synthesized expressions drop from tens of nodes under GP to roughly 5–8 nodes under NEON, indicating the learned guidance reduces program bloat.
  • A hybrid that applies grafting to half the population and conventional crossover to the other half (NEON-HH) is competitive with, and sometimes slightly better than, pure NEON at larger population sizes.
  • Because the GNN is trained once on synthetic data and then used across all 97 test problems, the saliency map transfers between problem instances without retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's proxy for prospective usefulness—presence in the upper part of a synthetic target tree—could be swapped for other criteria, such as semantic diversity or expected effect on the fitness gradient, which would isolate whether the proxy or the GNN architecture drives the gains.
  • The random ablated variant winning at population 1000 hints that when the library is large, diversity of stored subprograms can matter more than learned guidance; a natural extension is to scale the selection strategy with population size.
  • The same graph representation could support other informed operators, such as mutation targeting or semantic crossover point selection, without retraining the GNN.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes NEON, a semantics-aware search operator for tree-based genetic programming (GP) in symbolic regression. NEON maintains a library of subprograms generated by expanding subtrees selected from the population by one level, uses a graph neural network (GNN) to produce a saliency map over candidate expansion nodes, and grafts the top-scoring subprograms into offspring. The GNN is trained on synthetic expression trees to classify an expanded node as positive iff it appears in the upper part of a sampled target tree. The method is evaluated on 97 AI Feynman problems at four population sizes, comparing NEON, a hybrid NEON-HH, conventional GP, and an ablated version with random selection (NEON-Abl). The paper reports higher success rates for NEON than GP and NEON-Abl at most population sizes, and reports that successful NEON solutions are much smaller than GP solutions.

Significance. If the claimed improvements are robust, NEON would be a significant contribution: it learns a search operator from data, exploits semantic information without evaluating fitness of library entries, guarantees syntactic correctness by construction, and could port to other tree/graph domains. The paper also provides a clean ablation (NEON-Abl) that isolates the contribution of GNN guidance. These strengths make the research direction worth pursuing. However, the empirical support is currently preliminary: the evaluation uses a single run per problem, lacks statistical testing, and shows a reversal at the largest population size. The contribution is better framed as a proof-of-concept than as an established improvement.

major comments (4)
  1. [Sec. 4.4, Table 1] The central claim that NEON outperforms the ablated variant is not established because each configuration is run only once per problem and no significance tests are reported. With 97 binary outcomes per configuration, the difference at population 500 (NEON 0.1959 vs NEON-Abl 0.1856, i.e., one problem) is within sampling noise, and at population 1000 NEON-Abl (0.2268) beats NEON (0.1959) and ties NEON-HH (0.2062). The paper should report repeated runs with different random seeds and a paired statistical test (e.g., McNemar's test over the 97 problems) for each population size, or explicitly retract the claim of superiority over the ablation at population 1000.
  2. [Sec. 2.3.3, footnote 2] The selection rule (threshold 0.5, top k=5) was chosen because it 'yielded the best results' on the same benchmark suite, without a separate validation set. This creates selection bias that is load-bearing for the comparison against NEON-Abl, since the random ablation uses the same k and no tuned threshold. The absence of a validation split undermines the claim that the GNN guidance, rather than the tuned selection parameters, explains the observed differences. The authors should either fix the selection parameters a priori based on the training/validation set or apply the same tuning process to the random ablation.
  3. [Sec. 2.4.3 and Sec. 4.1] The GNN's training objective treats an expanded node as positive iff it occurs in the upper part of a synthetically sampled target tree. This is a proxy for prospective usefulness during evolutionary search, but the paper provides no validation that this proxy correlates with eventual success. The reversal at population 1000, where random selection outperforms the GNN, is consistent with the concern that GNN guidance may be no more informative than random for large libraries. The authors should add an analysis of whether GNN-selected subprograms actually contribute to successful solutions (e.g., measure how often library subprograms from GNN selection appear in the best-of-run programs) or compare against a baseline with an alternative GNN training signal.
  4. [Sec. 4.4, Table 3] The claim that NEON produces much smaller solutions than GP is based only on successful runs, but success rates vary across configurations and populations, and the number of successful runs is small (e.g., GP arity 2 at population 100 has 5 successes). The reported average sizes are therefore computed over different sets of problems and could be confounded by which problems are solved. The paper should report solution sizes with error bars or a statistical comparison over the matched set of problems solved by both configurations, or at least disclose the number of successes underlying each average.
minor comments (6)
  1. [Sec. 1] There is a typo: 'thy were manually designed' should be 'they were manually designed'.
  2. [Sec. 3] The phrase 'inconsistent withe the iterative' should be 'inconsistent with the iterative'.
  3. [Sec. 2.4.1] It is unclear which embedding is 'set to zero' for non-value nodes; the sentence should specify that both the 32-bit value embedding and the 32-bit target-difference embedding are zero for non-value nodes.
  4. [Sec. 2.2.2 and Fig. 1] The paper states that the expander draws 'a fraction of candidate solutions' and later uses 20%; the 'n=5' and 'top k=5' parameters are both introduced without explicit cross-referencing, which may confuse the reader about which step uses which value.
  5. [Sec. 4.3] The paper states 'over 1,500 runs in total,' which is arithmetically correct (4 configurations × 4 population sizes × 97 problems = 1,552 runs), but the phrasing is vague; please state the exact number.
  6. [General] The paper does not include a data or code availability statement. Given the 12-hour GNN training time, providing the trained model or implementation would substantially aid reproducibility.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the learned operator is trained on a synthetic reconstruction task and evaluated on an external benchmark without re-fitting.

full rationale

The central derivation is self-contained. The GNN is trained (Sec. 2.4.3) to classify one-step expansions of sampled subtrees according to whether the expansion occurs in the upper part of a synthetically sampled target expression; this is a standalone reconstruction objective, not a fit to the evaluation benchmark. The trained GNN is then frozen and applied across all AI Feynman runs (Secs. 4.1 and 4.3), and no constants or parameters of the method are fitted to the test problems. The self-citations (e.g., [15], [21]) are contextual related work and are not load-bearing for the success-rate comparison. The acknowledged choice of the best selection strategy in footnote 2 and the single-run-per-problem results in Table 1 are empirical-validity limitations, but they do not make any reported result equivalent to its own input by construction.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the GNN's learned weights and on the selection hyperparameters. The GNN weights are fitted to a synthetic expression distribution, not derived from first principles. The threshold and top-k were tuned on the benchmark, while the transferability of the synthetic training prior to AI Feynman is assumed. No new physical or conceptual entities are introduced.

free parameters (5)
  • top-k selection count k = 5
    Chosen in Sec 2.3.3 with footnote stating alternatives were tested and this yielded the best results, indicating tuning on the benchmark.
  • saliency threshold = 0.5
    Expanded nodes with saliency below 0.5 are discarded (Sec 2.3.3); selected by experiment rather than derived.
  • population sampling ratio = 20%
    The expander draws 20% of population programs to control cost (Sec 2.2.2); an arbitrary constant.
  • GNN architecture hyperparameters = 3 GAT layers, 4 heads, 256 hidden dims, ELU, learning rate 0.001
    Set in Sec 2.4.2 and 4.1 by design choice; no ablation justifies these exact values.
  • GNN trained weights = trained on synthetic expressions (not reported)
    The saliency signal depends entirely on the learned weights, which are fitted to a synthetic distribution of expressions (Sec 4.1); these weights determine the central behavior of NEON.
assumptions (3)
  • domain assumption The GNN trained on expressions with inputs drawn from a normal distribution will transfer to AI Feynman problems with physical variable ranges.
    Sec 4.1 generates training data with x ~ normal; Sec 4.3 applies to AI Feynman, which has physical constants and varied ranges. No transfer check is provided.
  • ad hoc to paper The binary classification target (an expansion is positive iff it occurs in the upper part of a known target tree) is a valid proxy for prospective usefulness in search.
    Sec 2.4.3 defines the training objective; this is a specific design choice not externally validated.
  • ad hoc to paper Averaged saliency over all training examples, thresholded at 0.5, identifies expansions likely to be useful in later generations.
    Sec 2.3.3 implements this selection rule; it is a heuristic, and the threshold was tuned.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Semantics-aware Search Operators for Genetic Programming." pith.science (2026). https://pith.science/paper/KBRR7UG6

@misc{pith2026250204568,
  author       = {Pith},
  title        = {Pith review of: Learning Semantics-aware Search Operators for Genetic Programming},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KBRR7UG6}},
  note         = {Machine review of arXiv:2502.04568}
}
read the original abstract

Fitness landscapes in test-based program synthesis are known to be extremely rugged, with even minimal modifications of programs often leading to fundamental changes in their behavior and, consequently, fitness values. Relying on fitness as the only guidance in iterative search algorithms like genetic programming is thus unnecessarily limiting, especially when combined with purely syntactic search operators that are agnostic about their impact on program behavior. In this study, we propose a semantics-aware search operator that steers the search towards candidate programs that are valuable not only actually (high fitness) but also only potentially, i.e. are likely to be turned into high-quality solutions even if their current fitness is low. The key component of the method is a graph neural network that learns to model the interactions between program instructions and processed data, and produces a saliency map over graph nodes that represents possible search decisions. When applied to a suite of symbolic regression benchmarks, the proposed method outperforms conventional tree-based genetic programming and the ablated variant of the method.

Figures

Figures reproduced from arXiv: 2502.04568 by the authors.

Figure 1
Figure 1. An example of graph expansion performed by [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The architecture of the GNN used for saliency estimation in NEON; [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 13 canonical work pages

  1. [1]

    Andrei Bajurnow and Vic Ciesielski. 2004. Layered Learning for Evolving Goal Scoring Behavior in Soccer Players. In Proceedings of the 2004 IEEE Congress on Evolutionary Computation. IEEE Press, Portland, Oregon, 1828–1835. doi:doi: 10.1109/CEC.2004.1331118 Learning Semantics-aware Search Operators for Genetic Programming

    work page Pith review arXiv 2004

  2. [2]

    Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow

    Matej Balog, Alexander L. Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. 2016. DeepCoder: Learning to Write Programs. arXiv preprint arXiv:1611.01989 (November 2016). https://arxiv.org/abs/1611.01989

    arXiv 2016

  3. [3]

    Luca Biggio, Tommaso Bendinelli, Alexander Neitz, Aurelien Lucchi, and Giambattista Parascandolo. 2021. Neural Symbolic Regression that Scales. arXiv:2106.06427 (June 2021). doi:10.48550/arXiv.2106.06427 arXiv:2106.06427 [cs]

    work page Pith review arXiv doi:10.48550/arxiv.2106.06427 2021

  4. [4]

    Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. 2015. Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289 (2015)

    arXiv 2015

  5. [5]

    Félix-Antoine Fortin, François-Michel De Rainville, Marc-André Gardner, Marc Parizeau, and Christian Gagné. 2012. DEAP: Evolutionary Algorithms Made Easy. Journal of Machine Learning Research 13 (jul 2012), 2171–2175

    work page 2012

  6. [6]

    Artur d’Avila Garcez and Luis C. Lamb. 2020. Neurosymbolic AI: The 3rd Wave. arXiv:2012.05876 (Dec. 2020). doi:10.48550/arXiv.2012.05876 arXiv:2012.05876 [cs]

    arXiv doi:10.48550/arxiv.2012.05876 2020

  7. [7]

    Pascal Hitzler and Md Kamruzzaman Sarker. 2022. Neuro-Symbolic Artificial Intelligence - The State of the Art. Number 342 in Frontiers in Artificial Intelligence and Applications. IOS Press, Amsterdam. https://www.iospress.com/catalog/ books/neuro-symbolic-artificial-intelligence-the-state-of-the-art

    work page 2022

  8. [8]

    Hornby and Jordan B

    Gregory S. Hornby and Jordan B. Pollack. 2002. Creating High-Level Components with a Generative Representation for Body-Brain Evolution.Artif. Life 8, 3 (2002), 223–246. doi:doi:10.1162/106454602320991837

    work page doi:10.1162/106454602320991837 2002

Show all 22 references
  1. [9]

    Pierre-Alexandre Kamienny, Stéphane d’Ascoli, Guillaume Lample, and François Charton. 2022. End-to-end symbolic regression with transformers. arXiv:2204.10532 [cs] (April 2022). http://arxiv.org/abs/2204.10532 arXiv: 2204.10532

    2022 arXiv

  2. [10]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Opti- mization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/...

    2015 arXiv

  3. [11]

    Pawel Liskowski, Iwo Bladek, and Krzysztof Krawiec. 2018. Neuro-guided genetic programming: prioritizing evolutionary search with neural networks. In GECCO ’18: Proceedings of the Genetic and Evolutionary Computation Con- ference, Hernan Aguirre et al. (Ed.). ACM, Kyoto, Japan...

    2018

  4. [12]

    Paweł Liskowski, Krzysztof Krawiec, and Nihat Engin Toklu. 2020. Neuromemetic Evolutionary Optimization. In Parallel Problem Solving from Nature – PPSN XVI: 16th International Conference, PPSN 2020, Leiden, The Netherlands, September 5-9, 2020, Proceedings, Part I (Leiden, The...

    2020 doi

  5. [13]

    Pawel Liskowski, Krzysztof Krawiec, Nihat Engin Toklu, and Jerry Swan. 2020. Program Synthesis as Latent Continuous Optimization: Evolutionary Search in Neural Embeddings. In Proceedings of the 2020 Genetic and Evolutionary Computa- tion Conference (GECCO ’20), Carlos Artemio ...

    2020

  6. [14]

    Aaron et al. Meurer. 2017. SymPy: symbolic computing in Python.PeerJ Computer Science 3 (Jan. 2017), e103. doi:10.7717/peerj-cs.103

    2017 doi

  7. [15]

    Pawlak, Bartosz Wieloch, and Krzysztof Krawiec

    Tomasz P. Pawlak, Bartosz Wieloch, and Krzysztof Krawiec. 2015. Semantic Backpropagation for Designing Search Operators in Genetic Programming. IEEE Transactions on Evolutionary Computation 19, 3 (June 2015), 326–340. doi:doi: 10.1109/TEVC.2014.2321259

    2015

  8. [16]

    Rosca and Dana H

    Justinian P. Rosca and Dana H. Ballard. 1996. Discovery of Subroutines in Genetic Programming. In Advances in Genetic Programming 2 , Peter J. Angeline and K. E. Kinnear, Jr. (Eds.). MIT Press, Cambridge, MA, USA, Chapter 9, 177–201. doi:doi:10.7551/mitpress/1109.003.0014

    1996 doi

  9. [17]

    Conor Ryan, Maarten Keijzer, and Mike Cattolico. 2004. Favorable Biasing of Function Sets Using Run Transferable Libraries. In Genetic Programming Theory and Practice II, Una-May O’Reilly, Tina Yu, Rick L. Riolo, and Bill Worzel (Eds.). Springer, Ann Arbor, Chapter 7, 103–120....

    2004 doi

  10. [18]

    Simari, Bowen Xi, and Lahari Pokala

    Paulo Shakarian, Chitta Baral, Gerardo I. Simari, Bowen Xi, and Lahari Pokala

  11. [19]

    Silviu-Marian Udrescu and Max Tegmark. 2020. AI Feynman: A physics-inspired method for symbolic regression. Science Advances 6, 16 (April 2020), eaay2631. doi:10.1126/sciadv.aay2631

    2020 doi

  12. [20]

    Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In International Conference on Learning Representations

    2018

  13. [21]

    Piotr Wyrwiński and Krzysztof Krawiec. 2024. Guiding Genetic Programming with Graph Neural Networks. In Proceedings of the Genetic and Evolutionary Computation Conference Companion (Melbourne, VIC, Australia) (GECCO ’24 Companion). Association for Computing Machinery, New York...

    2024

  14. [2023]

    Springer

    Neuro Symbolic Reasoning and Learning . Springer. doi:10.1007/978-3-031- 39179-8

    doi

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.