Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Fitness Landscape of Large Language Model-Assisted Automated Algorithm Search

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLM-assisted algorithm search operates on rugged, multimodal fitness landscapes that vary by task and model.

desk verdict A useful first map of what LLM-based algorithm search actually walks, but the headline claims about ruggedness and multimodality outrun the evidence because the graph is built from the search itself. read the letter →

arxiv 2504.19636 v3 pith:Q7CTUYJO submitted 2025-04-28 cs.AI cs.NE

Fei Liu , Qingfu Zhang , Jialong Shi , Xialiang Tong , Kun Mao , Mingxuan Yuan This is my paper · ORCID
classification cs.AIcs.NE
keywords fitnesslandscapelargelanguagemodelsautomatedalgorithmdesignevolutionarysearchgraph-basedanalysissimilaritycombinatorialoptimizationtrajectorynetworks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to characterize the fitness landscape faced by LLM-assisted automated algorithm search (LAS): the space of candidate algorithms that an LLM can generate and evolve. It claims that these landscapes are highly multimodal and rugged, particularly for combinatorial optimization heuristic design, with many local optima of similar quality, no funnel toward a single best region, and sparse connectivity between solutions. It further claims that the landscape structure differs across the six tested algorithm-design tasks and six LLMs, and that code-similarity measures (BLEU, weighted BLEU, AST match, dataflow match) correlate only weakly with performance differences. A reader should care because these properties determine whether and how LLM-based search can succeed: if the landscape is a field of disconnected peaks, then operators and models that escape local optima matter more than those that refine a single basin.

What carries the argument

The central object is the graph-based fitness landscape $G=(N,E)$, where each node is a candidate algorithm generated during one search run and each undirected edge records that one algorithm was generated from another via an LLM operator; edges are weighted by transition frequency. Because there is no natural distance in code space, the neighborhood relation is defined procedurally ($a_i$ is a neighbor of $a_j$ if $a_i = \mathrm{LLM}(\mathrm{Prompt}(a_j))$), and landscape properties are read off graph metrics, such as density, average degree, and clustering coefficient, along with fitness distributions. The argument also relies on four code-similarity measures (BLEU, weighted BLEU, AST match, dataflow match) used to relate code distance to performance delta and to operator behavior, with dataflow match treated as the most behaviorally meaningful.

What would settle it

Re-run the same LAS search but also generate a control set of algorithms by prompting the LLM independently (without a parent), then compare graph density, average degree, and clustering of the trajectory graph against a control graph built from random algorithm pairs; if the control shows similar low connectivity and multimodal fitness values, the landscape claims are artifacts of sampling rather than properties of the space.

Watch

Extended reading notes

Core claim

The paper's central claim is that the fitness landscape of LAS, built from a graph whose nodes are candidate algorithms produced during search and whose edges are parent-offspring transitions, is highly multimodal and rugged, especially in combinatorial optimization tasks, and that its structure varies across tasks and LLMs. Formally, the landscape is the triplet $(S, NS, f)$ with $f$ the mean performance over instances, and the neighborhood relation $NS$ defined by $a_i \in NS(a_j)$ iff $a_i$ is generated from $a_j$ by an LLM operator. Measured on that graph, the paper reports multiple approximate local optima with similar fitness, no funnel structure, and low average degree and clustering coefficient, together with fitness distributions in which roughly 80% of heuristic-design algorithms sit in the lowest 20% of fitness, the Mountain Car task is bimodal, and symbolic regression leaves 98% of algorithms far from optimal. Across LLMs on TSP, graph density, average degree, and clustering coefficient differ markedly, showing different exploration-exploitation balances. Finally, the four code-based similarity metrics correlate with each other and with performance delta, with AST and dataflow matching correlating most strongly on TSP, but the correlations are small and variable, so the paper concludes that current code-similarity metrics are not adequate for LAS and that specialized ones are needed.

Load-bearing premise

The paper treats the graph of algorithms actually generated during one search run, and the parent-offspring transitions between them, as an adequate representation of the underlying fitness landscape; if that trajectory sample is not representative of the algorithm space, the reported ruggedness and connectivity findings do not follow.

Editorial extensions

If this is right

  • If the landscape has no funnel, then search methods that only refine around the current best will plateau; escaping many equal-quality local optima becomes the central challenge.
  • Different LLMs produce measurably different landscape connectivity, so model choice is a search-strategy choice: dense, clustered landscapes favor local refinement, while sparse ones favor global exploration.
  • Task differences imply that the same LAS method should not be expected to behave uniformly; combinatorial heuristic tasks concentrate most algorithms in low-fitness regions, while symbolic regression leaves almost all algorithms far from optimum.
  • Because code similarity correlates with performance delta only weakly, distance-based mechanisms such as diversity maintenance or niching cannot rely on current metrics; LAS-specific similarity measures are needed.
  • Larger population sizes broaden exploration and reveal more landscape complexity, so population size is a practical lever for balancing exploitation and exploration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reported ruggedness is intrinsic, an immediately testable design rule is to favor mutation operators that make large syntactic jumps (low dataflow match) early in search, and to reserve semantically preserving operators for late refinement.
  • The graph metrics themselves could serve as a cheap diagnostic: an early run's density and clustering might predict whether a given LLM-task pair will get trapped, enabling adaptive operator or model selection mid-search.
  • The similarity-metric results suggest that an execution-trace-based or learned functional embedding of algorithms would outperform all four tested metrics at predicting performance delta, and such a metric could be validated directly on the paper's recorded trajectories.
  • A null-model comparison, randomly generated algorithms with the same number of nodes, would sharpen whether 'low connectivity' is a property of the algorithm space or an artifact of the trajectory sampling; that comparison is not in the paper but is directly computable from its data.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a graph-based approach to characterize the fitness landscape of LLM-assisted Algorithm Search (LAS). In this representation, nodes are candidate algorithms generated during an evolutionary search (specifically EoH), and edges are parent-offspring transitions induced by LLM-based operators. The authors run experiments across six algorithm design tasks and six LLMs, visualize the resulting graphs in 2D and 3D, compute graph metrics (density, average degree, clustering coefficient), and plot fitness distributions. They conclude that LAS landscapes are highly multimodal and rugged, lack a funnel structure, and exhibit low connectivity, with structural variation across tasks and LLMs. They additionally evaluate four code-based similarity metrics (BLEU, weighted BLEU, AST Match, Dataflow Match) and report correlations with performance deltas and operator type. The paper's central structural claims rest on interpreting these search-trajectory graphs as faithful samples of the underlying algorithm space.

Significance. If supported, the paper would provide a useful first characterization of LLM-driven algorithm search landscapes and inform the design of better search operators. The empirical scope is broad: six tasks spanning combinatorial optimization, reinforcement learning, and symbolic regression, together with six LLMs, and the authors position their work relative to existing code-evolution-graph studies. The similarity-metric analysis is a reasonable, though preliminary, contribution. However, the central claims about multimodality, ruggedness, absence of funnels, and low connectivity are methodologically undermined because the graph is built from a single search trajectory, not from an independent or representative sample of the algorithm space. Without repeated runs, null-model comparisons, or quantitative landscape measures, the main conclusions are not established. The paper does not provide machine-checked proofs or a reproducible code repository in the text, but it does build on the open-source LLM4AD platform, which is a strength.

major comments (4)
  1. [Section III] The graph-based representation defines the neighborhood as ai in NS(aj) if ai is generated from aj during the search, and nodes are only algorithms that the search happened to generate. This constructs a Search Trajectory Network, not a fitness landscape graph over the solution space S. As a result, the graph properties measured in Sections IV.B and IV.C are properties of the search process and the LLM's conditional generation distribution, not properties of the static algorithm space. To support claims about the underlying landscape, the authors must either reframe the paper as an analysis of search trajectory networks or supplement the construction with random or systematic sampling of the algorithm space and a null model that accounts for the search process.
  2. [Sections IV.B and IV.C (Table I)] The claims of 'low connectivity,' 'absence of funnel structure,' and 'multimodality' rest on visual inspection of 2D/3D layouts and on graph metrics that are directly shaped by the evolutionary search dynamics. For example, the EoH update step keeps a population of 20, and the run evaluates up to 2,000 candidates, so the average degree and density are bounded by the number of parents retained and the frequency of repeated transitions, not by the connectivity of the algorithm space itself. Without a null model (e.g., random generation of the same number of algorithms without selection, or a random graph with the same node set) or quantitative landscape measures (e.g., fitness-distance correlation, autocorrelation, local optima networks), the paper's central structural conclusions do not follow.
  3. [Section IV.A (Experimental Settings)] The experimental section describes a single run per task and LLM combination; no random seeds, repetitions, or statistical aggregation are mentioned. Single-run results cannot support statements such as 'the fitness landscape of LLM-assisted algorithm search is usually multi-modal with distinct patterns and characteristics across different algorithm design tasks and LLMs' (Section I). Repeated runs with different seeds are necessary to distinguish robust landscape properties from run-specific variability, and the paper currently provides no evidence that the observed structural differences across tasks and LLMs are reproducible.
  4. [Section IV.B, Figs. 3-7] The 2D and 3D 'landscape' visualizations appear to be graph layouts (the layout algorithm is not specified), whose coordinates have no defined relationship to any metric on the algorithm space. Visual observations such as 'absence of funnel structure' and the 'bimodal distribution' of the Car task are therefore not evidence about landscape geometry. Even if the visualizations are intended only as illustrations, the paper uses them as primary evidence for structural claims. Quantitative measures computed on a properly sampled space, with error bars across runs, are needed before making claims about multimodality or the presence or absence of funnels.
minor comments (5)
  1. [Section III, last paragraph] The sentence 'Since their are no well-defined distance metrics or neighbourhood structure' contains a typo ('their' should be 'there'), and the preceding claim that 'the neighbourhood is instead characterized by correlations between algorithms' does not match the generation-based definition of NS(aj) given immediately below.
  2. [Table I] The table reports density, average degree, and clustering coefficient, but not the number of nodes or edges for each LLM. Without these counts, the density values (e.g., 0.00069 vs. 0.00223) are hard to interpret; please add the graph sizes.
  3. [Section IV.B] The bullet stating that 'approximately 80% of the algorithms fall within the lowest 20% of fitness values' is ambiguous in a minimization setting, where lower values are better; please clarify whether 'lowest' refers to the best or the worst fitness values.
  4. [Section IV.F.e] The limitation paragraph says the analysis reveals 'small correlation coefficients to performance delta,' which appears to contradict the earlier statement in Section IV.F.a that Dataflow Match and AST Match yield 'the highest correlations with performance delta' for TSP. Please reconcile these two statements by reporting the actual coefficient values.
  5. [Fig. 8] The 5x5 matrix of subplots is very dense and difficult to read in the printed version; consider enlarging the figure or moving it to supplementary material, and explicitly state what the regression lines and shaded areas represent in the caption.

Circularity Check

1 steps flagged · score 5.0 of 10

The graph-based 'landscape' is built from a single EoH search trajectory, so the low-connectivity and density findings are forced by graph construction and the evaluation budget; multimodal fitness distributions are measured, not circular.

  1. self definitional [Section III (Graph-Based Representation of Fitness Landscape, node/edge definitions) and Section IV.B (Landscape on Different Tasks, Low Connectivity observation); also Table I in Section IV.C.]
    "Edges in the graph are undirected and represent the genetic relationship between a parent algorithm and its offspring. Specifically, an edge exists from algorithm ai to algorithm aj if aj is directly derived from ai through the application of LLM-based search operators, i.e., aj = LLM(Prompt(ai)). [Sec. III] The graph representation of the search landscape demonstrates low connectivity, as evidenced by the low average degree and sparsity of connections. [Sec. IV.B]"

    The landscape graph is not a sample of the algorithm space; it is the search trajectory itself. Nodes are only algorithms the search generated, and edges exist only when one algorithm was actually produced from another by the LLM prompt operator. With at most 2,000 evaluated candidates per run (population 20), the number of distinct transition edges is capped by the evaluation budget, so density = |E|/(n(n-1)/2) is forced to be very small for any LLM or task. Calling this 'low connectivity' of the landscape therefore restates the graph construction and budget rather than measuring connectivity of the underlying algorithm space.

full rationale

The paper's central derivation chain defines the fitness landscape graph as the search trajectory: nodes are algorithms produced during EoH runs and edges are actual parent-offspring LLM transitions. The subsequent structural findings (low density, low average degree, low clustering, and 'low connectivity') are then read as properties of the algorithm space. This is circular in the limited but concrete sense that the graph construction itself guarantees sparse edges: with a budget of 2,000 evaluations and edges representing only realized transitions, any run necessarily yields a low-density graph, so the 'low connectivity' observation does not independently characterize the underlying landscape. The multimodality claim, by contrast, rests on measured fitness distributions of the generated algorithms and is not definitionally forced; it may be biased by sampling and selection pressure, but that is a methodological limitation rather than a circular reduction. The similarity-metric analysis in Section IV.F is also empirically grounded and includes an explicit limitations paragraph, so it is not circular. Self-citations to EoH and LLM4AD supply the experimental framework and platform, but they do not by themselves determine the landscape conclusions, and the paper openly builds on Search Trajectory Networks rather than hiding that ancestry. Overall, one significant structural sub-claim reduces by construction, while the fitness-distribution and similarity results retain independent empirical content, yielding a partial circularity score of 5.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The report rests on four domain assumptions: the search-trajectory graph represents the landscape, the instance-set fitness scalarization is adequate, the four code-similarity metrics are meaningful proxies, and single runs are sufficient. The first and fourth are the most fragile because they directly shape the structural conclusions. No free parameters are fitted to data, and no new entities are introduced.

assumptions (4)
  • domain assumption The set of algorithms generated during a single EoH run, with parent-offspring transitions as neighborhoods, adequately represents the fitness landscape of the algorithm search space.
    Section III defines the landscape graph exclusively from search trajectories; all structural metrics in Sections IV.B and IV.C inherit this assumption.
  • domain assumption Fitness evaluated on the chosen instance sets is a sufficient objective for comparing algorithms.
    Section II-C defines F(a,I) as average over instances; the landscape conclusions depend on this scalarization.
  • domain assumption Code similarity metrics (BLEU, weighted BLEU, AST match, data-flow match) capture meaningful algorithmic similarity.
    Section IV.F applies these metrics to algorithm pairs; the paper itself notes they correlate weakly with performance, so this axiom is partially contested internally.
  • ad hoc to paper A single run per task/LLM combination suffices for landscape characterization.
    Section IV.A describes runs but reports no repetition or variance; landscape claims are drawn from these un-replicated runs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fitness Landscape of Large Language Model-Assisted Automated Algorithm Search." pith.science (2026). https://pith.science/paper/Q7CTUYJO

@misc{pith2026250419636,
  author       = {Pith},
  title        = {Pith review of: Fitness Landscape of Large Language Model-Assisted Automated Algorithm Search},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q7CTUYJO}},
  note         = {Machine review of arXiv:2504.19636}
}
read the original abstract

Using Large Language Models (LLMs) in an evolutionary or other iterative search framework have demonstrated significant potential in automated algorithm design. However, the underlying fitness landscape, which is critical for understanding its search behavior, remains underexplored. In this paper, we illustrate and analyze the fitness landscape of LLM-assisted Algorithm Search (LAS) using a graph-based approach, where nodes represent algorithms and edges denote transitions between them. We conduct extensive evaluations across six algorithm design tasks and six commonly-used LLMs. Our findings reveal that LAS landscapes are highly multimodal and rugged, particularly in combinatorial optimization tasks, with distinct structural variations across tasks and LLMs. Moreover, we adopt four different methods for algorithm similarity measurement and study their correlations to algorithm performance and operator behaviour. These insights not only deepen our understanding of LAS landscapes but also provide practical insights for designing more effective LAS methods.

Figures

Figures reproduced from arXiv: 2504.19636 by the authors.

Figure 1
Figure 1. Three distinct algorithm representations are commonly used in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. An illustration of a two-dimensional graph representation of fitness landscape, where nodes and edges are algorithms and their connections, respectively. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The 2D and 3D fitness landscapes for five algorithm design tasks: Online Bin Packing (OBP), Traveling Salesman Problem (TSP), Capacitated Vehicle [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: The scaled probability distribution of fitness values across six tasks. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The 2D fitness landscapes of TSP using different LLMs [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Visualization of landscapes with varying population sizes using the DeepSeek-V3 model on the TSP task. Each population size is depicted through [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: A directed network of candidate algorithms evolving toward the [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Matrix of metric relationships across TSP and OBP Tasks. It presents a 5x5 matrix of subplots visualizing the relationships between five key metrics: [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Distribution of performance delta by operator type. [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Dataflow Match Score distribution across operator types. [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: Correlation matrix between four similarity criteria and the perfor [PITH_FULL_IMAGE:figures/full_fig_p009_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PACE: Primitive-Aware Code Evolution for Automated Algorithm Design

    cs.SE 2026-08 conditional novelty 6.0 of 10

    PACE decouples reusable code components from host algorithms in LLM-based evolution, preserving them across generations, and reports stronger results than prior AAD methods on four tasks.

  2. A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving

    cs.NE 2025-09 conditional novelty 4.0 of 10

    A literature survey that classifies LLM-based optimization research into modeling and solving, with solving divided into LLMs as optimizers, low-level components, and high-level managers.

Reference graph

Works this paper leans on

43 extracted references · 29 canonical work pages · cited by 2 Pith papers

  1. [1]

    Evolution of heuristics: Towards efficient automatic algo- rithm design using large language model,

    F. Liu, T. Xialiang, M. Yuan, X. Lin, F. Luo, Z. Wang, Z. Lu, and Q. Zhang, “Evolution of heuristics: Towards efficient automatic algo- rithm design using large language model,” in Forty-first International Conference on Machine Learning , 2024

    work page 2024

  2. [2]

    Multi-objective evolution of heuristic using large language model,

    S. Yao, F. Liu, X. Lin, Z. Lu, Z. Wang, and Q. Zhang, “Multi-objective evolution of heuristic using large language model,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 39, no. 25, 2025, pp. 27 144–27 152

    work page 2025

  3. [3]

    Mathematical discoveries from program search with large language models,

    B. Romera-Paredes, M. Barekatain, A. Novikov, M. Balog, M. P. Kumar, E. Dupont, F. J. Ruiz, J. S. Ellenberg, P. Wang, O. Fawzi et al. , “Mathematical discoveries from program search with large language models,” Nature, vol. 625, no. 7995, pp. 468–475, 2024

    2024

  4. [4]

    Reevo: Large language models as hyper-heuristics with reflective evolution,

    H. Ye, J. Wang, Z. Cao, F. Berto, C. Hua, H. Kim, J. Park, and G. Song, “Reevo: Large language models as hyper-heuristics with reflective evolution,”Advances in neural information processing systems, vol. 37, pp. 43 571–43 608, 2024

    2024

  5. [5]

    Llamea: A large language model evolutionary algorithm for automatically generating metaheuristics,

    N. van Stein and T. B ¨ack, “Llamea: A large language model evolutionary algorithm for automatically generating metaheuristics,” IEEE Transac- tions on Evolutionary Computation , 2024

    2024

  6. [6]

    In-the-loop hyper-parameter optimization for llm-based automated design of heuristics,

    N. van Stein, D. Vermetten, and T. B ¨ack, “In-the-loop hyper-parameter optimization for llm-based automated design of heuristics,” ACM Trans- actions on Evolutionary Learning , 2024

    work page 2024

  7. [7]

    Evolutionary computation in the era of large language model: Survey and roadmap,

    X. Wu, S.-h. Wu, J. Wu, L. Feng, and K. C. Tan, “Evolutionary computation in the era of large language model: Survey and roadmap,” IEEE Transactions on Evolutionary Computation , 2024

    work page 2024

  8. [8]

    A systematic survey on large language models for algorithm design,

    F. Liu, Y . Yao, P. Guo, Z. Yang, X. Lin, X. Tong, M. Yuan, Z. Lu, Z. Wang, and Q. Zhang, “A systematic survey on large language models for algorithm design,” arXiv preprint arXiv:2410.14716 , 2024

    arXiv 2024

Show all 43 references
  1. [9]

    Understanding the importance of evolutionary search in automated heuristic design with large language models,

    R. Zhang, F. Liu, X. Lin, Z. Wang, Z. Lu, and Q. Zhang, “Understanding the importance of evolutionary search in automated heuristic design with large language models,” inInternational Conference on Parallel Problem Solving from Nature. Springer, 2024, pp. 185–202

    2024

  2. [10]

    Physics of language models: Part 3.1, knowl- edge storage and extraction,

    Z. Allen-Zhu and Y . Li, “Physics of language models: Part 3.1, knowl- edge storage and extraction,” arXiv preprint arXiv:2309.14316 , 2023

    2023 arXiv

  3. [11]

    A survey of techniques for characterising fitness landscapes and some possible ways forward,

    K. M. Malan and A. P. Engelbrecht, “A survey of techniques for characterising fitness landscapes and some possible ways forward,” Information Sciences, vol. 241, pp. 148–163, 2013

    2013

  4. [12]

    A survey of fitness landscape analysis for optimization,

    F. Zou, D. Chen, H. Liu, S. Cao, X. Ji, and Y . Zhang, “A survey of fitness landscape analysis for optimization,” Neurocomputing, vol. 503, pp. 129–139, 2022

    2022

  5. [13]

    Local optima networks: A new model of combinatorial fitness landscapes,

    G. Ochoa, S. Verel, F. Daolio, and M. Tomassini, “Local optima networks: A new model of combinatorial fitness landscapes,” in Recent advances in the theory and application of fitness landscapes . Springer, 2014, pp. 233–262

    2014

  6. [14]

    Homotopic convex transformation: A new landscape smoothing method for the traveling salesman problem,

    J. Shi, J. Sun, Q. Zhang, and K. Ye, “Homotopic convex transformation: A new landscape smoothing method for the traveling salesman problem,” IEEE Transactions on Cybernetics , vol. 52, no. 1, pp. 495–507, 2020

    2020

  7. [15]

    Fitness landscape optimization makes stochastic symbolic search by genetic programming easier,

    Z. Huang, Y . Mei, F. Zhang, M. Zhang, and W. Banzhaf, “Fitness landscape optimization makes stochastic symbolic search by genetic programming easier,” IEEE Transactions on Evolutionary Computation , 2025

    2025

  8. [16]

    Fitness landscape analysis of convolutional neural network architectures for image classification,

    N. M. Rodrigues, K. M. Malan, G. Ochoa, L. Vanneschi, and S. Silva, “Fitness landscape analysis of convolutional neural network architectures for image classification,” Information Sciences , vol. 609, pp. 711–726, 2022. 11

    2022

  9. [17]

    Landscape analysis based vs. domain-specific optimization for engineering design applications: A clear case,

    R. de Winter, F. X. Long, A. Thomaser, T. H. B ¨ack, N. van Stein, and A. V . Kononova, “Landscape analysis based vs. domain-specific optimization for engineering design applications: A clear case,” in 2024 IEEE Conference on Artificial Intelligence (CAI) . IEEE, 2024, pp. 776–781

    2024

  10. [18]

    On the effects of smoothing rugged landscape by different toy problems: A case study on ubqp,

    W. Wang, J. Shi, J. Sun, A. Liefooghe, Q. Zhang, and Y . Fan, “On the effects of smoothing rugged landscape by different toy problems: A case study on ubqp,” in 2024 IEEE Congress on Evolutionary Computation (CEC). IEEE, 2024, pp. 01–08

    2024

  11. [19]

    Stalling in space: Attractor analysis for any algorithm,

    S. L. Thomson, Q. Renau, D. Vermetten, E. Hart, N. v. Stein, and A. V . Kononova, “Stalling in space: Attractor analysis for any algorithm,” in International Conference on the Applications of Evolutionary Compu- tation (Part of EvoStar) . Springer, 2025, pp. 510–526

    2025

  12. [20]

    Code evolution graphs: Understanding large language model driven design of algo- rithms,

    N. van Stein, A. V . Kononova, L. Kotthoff, and T. B¨ack, “Code evolution graphs: Understanding large language model driven design of algo- rithms,” in Proceedings of the Genetic and Evolutionary Computation Conference, 2025, pp. 943–951

    2025

  13. [21]

    Hyper-heuristics: A survey of the state of the art,

    E. K. Burke, M. Gendreau, M. Hyde, G. Kendall, G. Ochoa, E. ¨Ozcan, and R. Qu, “Hyper-heuristics: A survey of the state of the art,” Journal of the Operational Research Society , vol. 64, pp. 1695–1724, 2013

    2013

  14. [22]

    Automated design of metaheuristic algorithms,

    T. St ¨utzle and M. L ´opez-Ib´a˜nez, “Automated design of metaheuristic algorithms,” in Handbook of metaheuristics . Springer, 2018, pp. 541– 579

    2018

  15. [23]

    Pillay and R

    N. Pillay and R. Qu, Automated Design of Machine Learning and Search Algorithms. Springer, 2021

    2021

  16. [24]

    Toward automated algorithm design: A survey and practical guide to meta-black-box- optimization,

    Z. Ma, H. Guo, Y .-J. Gong, J. Zhang, and K. C. Tan, “Toward automated algorithm design: A survey and practical guide to meta-black-box- optimization,” IEEE Transactions on Evolutionary Computation , 2025

    2025

  17. [25]

    Automated algorithm selection: Survey and perspectives,

    P. Kerschke, H. H. Hoos, F. Neumann, and H. Trautmann, “Automated algorithm selection: Survey and perspectives,” Evolutionary computa- tion, vol. 27, no. 1, pp. 3–45, 2019

    2019

  18. [26]

    Automl loss landscapes,

    Y . Pushak and H. Hoos, “Automl loss landscapes,” ACM Transactions on Evolutionary Learning , vol. 2, no. 3, pp. 1–30, 2022

    2022

  19. [27]

    Automated design of search algorithms: Learning on algorithmic components,

    W. Meng and R. Qu, “Automated design of search algorithms: Learning on algorithmic components,” Expert Systems with Applications, vol. 185, p. 115493, 2021

    2021

  20. [28]

    W. B. Langdon and R. Poli, Foundations of genetic programming . Springer Science & Business Media, 2013

    2013

  21. [29]

    Explainable artifi- cial intelligence by genetic programming: A survey,

    Y . Mei, Q. Chen, A. Lensen, B. Xue, and M. Zhang, “Explainable artifi- cial intelligence by genetic programming: A survey,” IEEE Transactions on Evolutionary Computation , vol. 27, no. 3, pp. 621–641, 2022

    2022

  22. [30]

    Survey on genetic programming and machine learning techniques for heuristic design in job shop scheduling,

    F. Zhang, Y . Mei, S. Nguyen, and M. Zhang, “Survey on genetic programming and machine learning techniques for heuristic design in job shop scheduling,” IEEE Transactions on Evolutionary Computation, 2023

    2023

  23. [31]

    Evolving code with a large language model,

    E. Hemberg, S. Moskal, and U.-M. O’Reilly, “Evolving code with a large language model,” Genetic Programming and Evolvable Machines , vol. 25, no. 2, p. 21, 2024

    2024

  24. [32]

    Fitness landscapes,

    P. F. Stadler, “Fitness landscapes,” in Biological evolution and statistical physics. Springer, 2002, pp. 183–204

    2002

  25. [33]

    A study of nk landscapes’ basins and local optima networks,

    G. Ochoa, M. Tomassini, S. V ´erel, and C. Darabos, “A study of nk landscapes’ basins and local optima networks,” in Proceedings of the 10th annual conference on Genetic and evolutionary computation, 2008, pp. 555–562

    2008

  26. [34]

    Search trajectory networks of population-based algorithms in continuous spaces,

    G. Ochoa, K. M. Malan, and C. Blum, “Search trajectory networks of population-based algorithms in continuous spaces,” in International Conference on the Applications of Evolutionary Computation (Part of EvoStar). Springer, 2020, pp. 70–85

    2020

  27. [35]

    Algorithm evolution using large language model,

    F. Liu, X. Tong, M. Yuan, and Q. Zhang, “Algorithm evolution using large language model,” arXiv preprint arXiv:2311.15249 , 2023

    2023 arXiv

  28. [36]

    Llm4ad: A platform for algorithm design with large language model,

    F. Liu, R. Zhang, Z. Xie, R. Sun, K. Li, X. Lin, Z. Wang, Z. Lu, and Q. Zhang, “Llm4ad: A platform for algorithm design with large language model,” arXiv preprint arXiv:2412.17287 , 2024

    2024

  29. [37]

    Openai gym,

    G. Brockman, “Openai gym,” arXiv preprint arXiv:1606.01540 , 2016

    2016 arXiv

  30. [38]

    LLM-SR: Scientific equation discovery via programming with large language models,

    P. Shojaee, K. Meidani, S. Gupta, A. B. Farimani, and C. K. Reddy, “LLM-SR: Scientific equation discovery via programming with large language models,” in The Thirteenth International Conference on Learn- ing Representations, 2025

    2025

  31. [39]

    CST-LLM: Enhancing airfoil parameterization method with large language model,

    K. Zheng, Y . Wang, F. Liu, Q. Zhang, and W. Song, “CST-LLM: Enhancing airfoil parameterization method with large language model,” Aerospace Science and Technology , p. 110548, 2025

    2025

  32. [40]

    Alphaevolve: A coding agent for scientific and algorithmic discovery,

    A. Novikov, N. V ˜u, M. Eisenberger, E. Dupont, P.-S. Huang, A. Z. Wagner, S. Shirobokov, B. Kozlovskii, F. J. Ruiz, A. Mehrabian et al., “Alphaevolve: A coding agent for scientific and algorithmic discovery,” arXiv preprint arXiv:2506.13131 , 2025

    2025 arXiv

  33. [41]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting of the Association for Computational Linguistics , 2002, pp. 311–318

    2002

  34. [42]

    Codebleu: a method for automatic evaluation of code synthesis,

    S. Ren, D. Guo, S. Lu, L. Zhou, S. Liu, D. Tang, N. Sundaresan, M. Zhou, A. Blanco, and S. Ma, “Codebleu: a method for automatic evaluation of code synthesis,” arXiv preprint arXiv:2009.10297 , 2020

    2009 arXiv

  35. [43]

    Graphcodebert: Pre-training code rep- resentations with data flow,

    D. Guo, S. Ren, S. Lu, Z. Feng, D. Tang, S. LIU, L. Zhou, N. Duan, A. Svyatkovskiy, S. Fu et al. , “Graphcodebert: Pre-training code rep- resentations with data flow,” in International Conference on Learning Representations

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.