Surrogate-Assisted Evolution for Efficient Multi-branch Connection Design in Deep Neural Networks

Daniel Garc\'ia N\'u\~nez; Edgar Galv\'an; Fergal Stapleton; Yanan Sun

REVIEW 4 major objections 4 minor 11 references

Surrogate-Assisted Evolution for Efficient Multi-branch Connection Design in Deep Neural Networks

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A semantic surrogate and pre-selection allow multi-branch DNN architectures to evolve competitively while cutting GPU time by 11.8–16.6%.

desk verdict A legitimate incremental extension of the authors' own NeuroLGP-SM work, but the headline efficiency-accuracy claim rests on an untested partial-training transfer assumption and four-run statistics. read the letter →

arxiv 2506.20469 v1 pith:4NLU3J4K submitted 2025-06-25 cs.NE

Fergal Stapleton , Daniel García Núñez , Yanan Sun , Edgar Galván This is my paper · ORCID

classification cs.NE

keywords neuroevolutionneuralarchitecturesearchlineargeneticprogrammingmulti-branchconnectionssurrogate-assistedevolutionaryalgorithmssemanticvectorsKrigingpartialleastsquarespre-selection

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multi-branch connections make deep networks more expressive but harder to search, and full training of every candidate is expensive. This paper argues that both problems can be eased at once: a Linear Genetic Programming encoding expresses multi-branch topologies, while a semantic surrogate model estimates fitness from partially trained networks. The proposed pre-selection variant, which initializes the surrogate with high-performing individuals, produced the best average test accuracy on BreakHis ×40 and BreakHis ×200 and cut GPU time by 11.8–16.6% relative to the fully trained evolutionary search. If the claim holds, surrogate-assisted neuroevolution can move from dozens or hundreds of evaluated points to thousands on modern DNNs.

What carries the argument

The machinery is a Linear Genetic Programming (LGP) encoding—a linear sequence of register-based instructions that builds a DNN graph in reverse—combined with a semantic distance: each network's output vector is the flattened final softmax layer over all test images times the number of classes. Because these vectors are fixed-length, architectures of different depth and branch count can be compared by ordinary distance, and Kriging Partial Least Squares (KPLS) can regress fitness onto them without maximum-likelihood estimation over huge parameter spaces. The pre-selection management strategy supplies the surrogate's initial training data by sampling the best partially trained individuals from a first generation rather than random individuals, and it carries the argument by bypassing genotype-based topology distances and allowing thousands of sample points.

What would settle it

Take one generation of evolved architectures, train each for 10 and 30 epochs, and compare the KPLS-predicted ranking with the full-training ranking; if Kendall's tau drops below roughly 0.5, or the pre-selected best architecture's full-training accuracy underperforms a randomly sampled baseline, then the time savings of Surrogate-PS do not transfer to deployment.

Watch

Extended reading notes

Core claim

The central discovery is that the semantic output of a network—the flattened softmax predictions over the test set—provides a fixed-length distance vector for surrogate modeling regardless of how many layers or branches a network has. NeuroLGP-MB encodes multi-branch topologies with Linear Genetic Programming, reading the genotype in reverse so that a CONCAT instruction marks a split; the evolutionary search therefore avoids expensive graph-edit genotypic distances. The proposed pre-selection surrogate pipeline splits the population (40% fully trained to 30 epochs, 60% partially trained to 10 epochs), feeds the semantic vectors of fully evaluated individuals into Kriging Partial Least Squares, estimates fitness of new individuals, and uses expected improvement for selection. Across four runs, this Surrogate-PS variant achieved the highest mean accuracy on BreakHis ×40 (0.919 ± 0.032) and BreakHis ×200 (0.939 ± 0.010), matched the other methods on Chest X-Ray, and cut GPU hours by 11.8–16.6% relative to the expensive approach. Elite-population analysis shows the search favors deeper networks with relatively few concatenation layers.

Load-bearing premise

The whole method depends on the assumption that a network trained for only 10 epochs—ranked through the KPLS surrogate—predicts which architectures would perform best after the full 30 epochs, and the reported rank correlations are moderate and vary by dataset.

Editorial extensions

If this is right

Multi-branch topologies can be compared for surrogate modeling using fixed-length semantic output vectors, so no graph-edit distance over variable-length genotypes is required.
Pre-selection with 10-epoch partial training provides a cheaper fitness estimate that preserves or improves mean accuracy, with 11.8–16.6% GPU-time savings over full training.
Surrogate-assisted neuroevolution can scale to thousands of evaluated sample points, moving beyond the dozens or hundreds typical of earlier surrogate neuroevolution.
The evolutionary search under this setup tends to produce deeper networks with relatively few branch/concatenation layers, an observation that can guide future mutation and crossover design.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Inference: If the partial-training ranking transfers to full training (Kendall's tau 0.63–0.74 in the paper), the same pre-selection scheme could lower cost in other neural architecture search spaces where genotype comparison is expensive, including graph-based architecture search.
Inference: The fixed-length semantic vector could double as an explicit diversity objective in multi-objective neuroevolution, not only as a distance for surrogate fitness.
Inference: Because the semantic vector's length grows with dataset size times class count, a testable extension is to use per-batch or class-conditional semantic summaries to keep the surrogate scalable to larger datasets.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a solid, incremental extension of the authors' own NeuroLGP-SM work, not a breakthrough. The genuinely new pieces are an LGP encoding for multi-branch DNNs and a pre-selection scheme that initializes the KPLS surrogate with high-performing individuals. The semantic-vector distance idea is real and it does let the surrogate scale to thousands of sample points, which is useful for DNN neuroevolution.

What the paper does well: the encoding is clever and the reverse-traversal construction for concat layers is clearly explained. The pre-selection idea is sensible—seed the surrogate with good individuals instead of random ones—and it does buy an 11.8–16.6% GPU-time reduction over full evolution while roughly holding accuracy on two of three datasets. The depth/complexity analysis of elite individuals is a nice extra. There is no circularity problem; the surrogate is fit to measured fitness values and used for standard model management.

The soft spots are real but addressable. The central claim that Sur-PS "produced the best networks on average" rests on four runs with overlapping standard deviations and no significance tests. On Chest X-Ray the expensive baseline has the highest mean accuracy. The abstract's claim that the advanced surrogate "outperforms baseline, computationally expensive, and simpler surrogate models" is not uniformly supported by Table 1: on BreakHis×200 the plain surrogate has better R² (0.937 vs 0.746). The biggest unaddressed risk, and the one I'd push on in review, is the transfer assumption: the surrogate is queried on semantic vectors from 10-epoch partial training, but the fitness target is 30-epoch accuracy. The paper never directly measures the correlation between epoch-10 and epoch-30 rankings, so we don't know whether the time saving comes at the cost of selection quality. That is testable and should be done. Also, the semantic vectors are built from the test set's outputs; the paper doesn't clarify whether test data leaks into surrogate training. No code or data is released, which makes the four-run statistics harder to evaluate.

Bottom line: the method is plausible, internally consistent, and honestly described. The flaws are not fatal. I'd send it to a serious referee, but the revision needs significance tests, a direct epoch-10-vs-epoch-30 rank-correlation analysis, and a clear statement on test-set usage. Worth a reading-group slot if you want to discuss surrogate model management in neuroevolution.

Referee Report

4 major / 4 minor

Summary. The paper proposes NeuroLGP-MB, a Linear Genetic Programming representation for encoding multi-branch connections in deep neural networks, and combines it with a Kriging Partial Least Squares surrogate model. A pre-selection variant, Sur-PS, initializes the surrogate with high-performing partially trained individuals and uses a 40/60 split between fully evaluated and partially trained candidates. The method is compared with a baseline random-search approach, a fully trained evolutionary approach, and a standard surrogate-assisted evolutionary approach on BreakHis×40, BreakHis×200, and Chest X-Ray datasets over four runs. The authors report that Sur-PS achieves the highest average accuracy on BreakHis×40 and BreakHis×200 while reducing GPU time by 11.8–16.6% relative to the expensive model, and they analyze elite-population depth and topological complexity.

Significance. If the empirical claims are supported, the paper offers a practically relevant contribution: an LGP encoding that sidesteps graph-edit distance for multi-branch topologies, semantic vectors that allow surrogate-assisted search over thousands of candidate networks, and a pre-selection strategy that improves search initialization. The paper also provides an explicit analysis of how elite depth and complexity evolve, which is useful for understanding search behavior. However, the current evidence base is narrow: four runs per condition, no significance tests, and an unvalidated transfer assumption between 10-epoch and 30-epoch training. The central efficiency claim depends on this transfer, so the paper's contribution is currently promising but not fully established.

major comments (4)

[Section 2.3 and Section 3.1] The central efficiency claim relies on the assumption that a 10-epoch partial training run ranks architectures the way a full 30-epoch training run would. In the model-management loop described in Section 2.3, annotations (i) and (ii), the KPLS surrogate is trained on fully evaluated individuals but queried on partially trained individuals. The paper never directly measures the correlation between an architecture's validation accuracy at epoch 10 and its accuracy at epoch 30. Table 1 reports KPLS fit quality (MSE, Kendall's Tau, R2) on the surrogate's training data, not the epoch-10-to-epoch-30 transfer. If partial-training rankings diverge from full-training rankings, the reported accuracy advantage of Sur-PS on BreakHis×40 and BreakHis×200, and the accompanying 11.8–16.6% GPU-time saving, would not transfer to real deployment. This should be tested directly, for example by reporting rank correlations between epoch-10 and epoch-30 accuracies over a sample of architectures on each dataset, or by comparing final selected networks against a full-training control.
[Section 4.1] The headline claim that Sur-PS 'outperforms' the baseline, expensive, and simpler surrogate models is not statistically supported. Each condition is evaluated with only four runs, no significance tests are reported, and the standard deviations overlap. For BreakHis×40, Sur-PS gives 0.919±0.032 versus 0.907±0.029 for the expensive model and 0.904±0.029 for the surrogate model; for BreakHis×200, Sur-PS gives 0.939±0.010 versus 0.930±0.030 for the expensive model. On Chest X-Ray, the expensive method has the highest mean accuracy (0.917±0.016 versus 0.914±0.014 for Sur-PS), directly contradicting the abstract's statement that the advanced surrogate outperforms the computationally expensive model. The paper should either provide significance tests, confidence intervals, or an explicit statement that the differences are not significant and that the contribution is efficiency rather than accuracy gains.
[Table 1 and Abstract] There is an internal inconsistency between the abstract's claim that the more advanced surrogate model 'outperforms baseline, computationally expensive, and simpler surrogate models' and the results in Table 1. On BreakHis×200, the standard surrogate has R2 of 0.9373 while Sur-PS has R2 of 0.7458; on BreakHis×40, the standard surrogate has slightly higher Kendall's Tau (0.6536 versus 0.6480) and R2 (0.6239 versus 0.6185). Only on Chest X-Ray are all three surrogate metrics better for Sur-PS. The claim of uniform superiority over the simpler surrogate is therefore not supported by the reported data. The abstract and conclusions should be tempered, or the surrogate-quality comparison should be analyzed separately from end-to-end accuracy and runtime.
[Section 3.1] The experimental setup is not specified precisely enough to establish a fair comparison across methods. Population size and generation count are stated only for the expensive approach (Pop. size = 50, Gen. size = 15); the surrogate and Sur-PS variants do not clearly state their population and generation budgets. The baseline 'random search' method also needs a clear statement of how many architectures it evaluates and whether it uses the same encoding and evaluation protocol. The time-analysis sentence mentions '150 individuals trained to 10 epochs' for Sur-PS, which implies a specific population size, but this is not connected to the stated population/generation parameters. Without this information, the reported runtime savings and accuracy comparisons cannot be independently assessed.

minor comments (4)

[Section 4.1] There are several typos: 'Chext X-Ray' should be 'Chest X-Ray', and 'Kendell's Tau' should be 'Kendall's Tau'.
[Equation (1)] Equation (1) contains an extra closing parenthesis: D(s(p_i), s(p_j))) should be D(s(p_i), s(p_j)).
[Figure 1] The table in Figure 1 has two 'I2' column headers; the second should presumably be 'I3'. Please clarify the register operand naming.
[Section 2.1] The KPLS method is referenced to [1] but no KPLS hyperparameters (e.g., number of PLS components, kernel parameters) are reported. Providing these would help reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the surrogate-assisted evolution pipeline is an empirical method whose claims are measured outcomes, not consequences of definitions.

full rationale

This is an empirical systems paper, not a derivation, and I find no step where a result is forced by definition or by a self-citation chain. The KPLS surrogate is trained on measured fitness values of fully evaluated individuals (Section 2.3, annotation (iii)-(iv)), and its predictions are used to guide search via Expected Improvement (annotation (v)); this is standard supervised model management rather than a self-referential prediction. The pre-selection variant (Sur-PS) initializes the surrogate with high-performing individuals, but the reported final accuracies are actual test accuracies from fully trained networks, not surrogate outputs, so the headline result is not forced by the surrogate construction. The time savings are measured GPU hours, not derived from the surrogate fit. The self-citations ([4], [5], [9], [10]) support background about semantics and prior NeuroLGP-SM work; they do not contain the central claim of this paper, and no load-bearing argument reduces to an unverified self-citation. Potential weaknesses such as the small number of runs, overlapping error bars, the untested transfer from 10-epoch partial training to 30-epoch ranking, and the abstract's overstatement about 'outperforming' all models are empirical and statistical concerns, not circularity. Therefore the appropriate score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method is empirical and rests on several hand-chosen experimental parameters and proxy assumptions rather than a derivation. No new physical or mathematical entities are introduced.

free parameters (5)

Population split for full vs partial evaluation = 40% full / 60% partial
Hand-chosen model management setting (Section 2.3, step i); no sensitivity analysis is given.
Partial training epochs = 10 epochs
Assumed proxy for full 30-epoch training (Section 2.3, Section 3.1); validity is load-bearing for the surrogate ranking.
Pre-selection sample size = 150 individuals
Number of briefly trained individuals used to initialize the Sur-PS surrogate (Section 4.1); the paper notes this overhead is substantial.
Evolutionary budget = population 50, generations 15
Fixed search budget for expensive/surrogate variants (Section 3.1); no budget sensitivity analysis.
KPLS hyperparameters = not specified
Number of PLS components and theta maximum-likelihood settings are delegated to reference [1] and not reported.

assumptions (4)

standard math Kriging spatial correlation and PLS dimension reduction assumptions apply to semantic output vectors
Surrogate fitness estimation relies on KPLS from [1] without validating its assumptions on this semantic space.
domain assumption Softmax semantic vectors on evaluation-set images capture architecture performance sufficiently for ranking
Section 2.1 uses these vectors as the sole input to the surrogate; the paper only measures correlation after fitting, not predictive independence.
domain assumption Ten-epoch partial training ranks architectures the same way full 30-epoch training would
Section 2.3 and Section 3.1 use 10-epoch training for 60% of the population; no comparison of partial vs full rankings is provided.
standard math Linear genetic programming effective-code interpretation yields valid DNN architectures
The graph construction in Section 2.2 follows LGP conventions from [2]; no formal verification of the decoding is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Surrogate-Assisted Evolution for Efficient Multi-branch Connection Design in Deep Neural Networks." pith.science (2026). https://pith.science/paper/4NLU3J4K

@misc{pith2026250620469,
  author       = {Pith},
  title        = {Pith review of: Surrogate-Assisted Evolution for Efficient Multi-branch Connection Design in Deep Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4NLU3J4K}},
  note         = {Machine review of arXiv:2506.20469}
}

read the original abstract

State-of-the-art Deep Neural Networks (DNNs) often incorporate multi-branch connections, enabling multi-scale feature extraction and enhancing the capture of diverse features. This design improves network capacity and generalisation to unseen data. However, training such DNNs can be computationally expensive. The challenge is further exacerbated by the complexity of identifying optimal network architectures. To address this, we leverage Evolutionary Algorithms (EAs) to automatically discover high-performing architectures, a process commonly known as neuroevolution. We introduce a novel approach based on Linear Genetic Programming (LGP) to encode multi-branch (MB) connections within DNNs, referred to as NeuroLGP-MB. To efficiently design the DNNs, we use surrogate-assisted EAs. While their application in simple artificial neural networks has been influential, we scale their use from dozens or hundreds of sample points to thousands, aligning with the demands of complex DNNs by incorporating a semantic-based approach in our surrogate-assisted EA. Furthermore, we introduce a more advanced surrogate model that outperforms baseline, computationally expensive, and simpler surrogate models.

Figures

Figures reproduced from arXiv: 2506.20469 by the authors.

**Figure 2.** Left: Diagram showing the interplay between a typical evolutionary algorithm and a surrogate model approach. Right: The surrogate model management strategy is shown on a more granular level. Top: Pre-selection approach used to initialise the surrogate model. thousand images. Training, validation, test set were used for network evaluation and a further test set was used for reporting (∼15% of dataset size). The ima… view at source ↗

**Figure 3.** Average number of layers for expensive, surrogate and surrogate-PS across 15 generation for each dataset. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

11 extracted references · 10 canonical work pages

[1]

Mohamed Amine Bouhlel, Nathalie Bartoli, Abdelkader Otsmane, and Joseph Morlier. 2016. Improving kriging surrogates of high-dimensional design models by Partial Least Squares dimension reduction. Structural and Multidisciplinary Optimization 53 (2016), 935–952

work page 2016
[2]

Markus Brameier and Wolfgang Banzhaf. 2007. Linear genetic programming. Vol. 1. Springer

work page 2007
[3]

Adam Gaier, Alexander Asteroth, and Jean-Baptiste Mouret. 2018. Data-efficient neuroevolution with kernel-based surrogate models. In Proceedings of the genetic and evolutionary computation conference. 85–92

work page 2018
[4]

Edgar Galván and Marc Schoenauer. 2019. Promoting semantic diversity in multi- objective genetic programming. In Proceedings of the Genetic and Evolutionary Computation Conference. 1021–1029

work page 2019
[5]

Edgar Galván, Leonardo Trujillo, and Fergal Stapleton. 2022. Semantics in multi- objective genetic programming. Applied Soft Computing 115 (2022), 108143

work page 2022
[6]

Daniel Kermany, Kang Zhang, Michael Goldbaum, et al. 2018. Labeled optical coherence tomography (oct) and chest x-ray images for classification. Mendeley data 2, 2 (2018), 651

work page 2018
[7]

Alberto Sanfeliu and King-Sun Fu. 1983. A distance measure between attributed relational graphs for pattern recognition. IEEE transactions on systems, man, and cybernetics 3 (1983), 353–362

work page 1983
[8]

Fabio Alexandre Spanhol, Luiz S Oliveira, Caroline Petitjean, and Laurent Heutte

work page

Show all 11 references

[9]

Fergal Stapleton, Brendan Cody-Kenny, and Edgar Galván. 2025. NeuroLGP-SM: A Surrogate-Assisted Neuroevolution Approach Using Linear Genetic Program- ming. In Optimization and Learning, Bernabé Dorronsoro, Martin Zagar, and El-Ghazali Talbi (Eds.). Springer Nature Switzerland,...

2025
[10]

Fergal Stapleton and Edgar Galván. 2024. NeuroLGP-SM: Scalable Surrogate- Assisted Neuroevolution for Deep Neural Networks. In 2024 IEEE Congress on Evolutionary Computation (CEC). 1–8. doi:10.1109/CEC60901.2024.10612039

2024 arXiv
[2016]

In 2016 international joint conference on neural networks (IJCNN)

Breast cancer histopathological image classification using convolutional neural networks. In 2016 international joint conference on neural networks (IJCNN). IEEE, 2560–2567

2016

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Mohamed Amine Bouhlel, Nathalie Bartoli, Abdelkader Otsmane, and Joseph Morlier. 2016. Improving kriging surrogates of high-dimensional design models by Partial Least Squares dimension reduction. Structural and Multidisciplinary Optimization 53 (2016), 935–952

work page 2016

[2] [2]

Markus Brameier and Wolfgang Banzhaf. 2007. Linear genetic programming. Vol. 1. Springer

work page 2007

[3] [3]

Adam Gaier, Alexander Asteroth, and Jean-Baptiste Mouret. 2018. Data-efficient neuroevolution with kernel-based surrogate models. In Proceedings of the genetic and evolutionary computation conference. 85–92

work page 2018

[4] [4]

Edgar Galván and Marc Schoenauer. 2019. Promoting semantic diversity in multi- objective genetic programming. In Proceedings of the Genetic and Evolutionary Computation Conference. 1021–1029

work page 2019

[5] [5]

Edgar Galván, Leonardo Trujillo, and Fergal Stapleton. 2022. Semantics in multi- objective genetic programming. Applied Soft Computing 115 (2022), 108143

work page 2022

[6] [6]

Daniel Kermany, Kang Zhang, Michael Goldbaum, et al. 2018. Labeled optical coherence tomography (oct) and chest x-ray images for classification. Mendeley data 2, 2 (2018), 651

work page 2018

[7] [7]

Alberto Sanfeliu and King-Sun Fu. 1983. A distance measure between attributed relational graphs for pattern recognition. IEEE transactions on systems, man, and cybernetics 3 (1983), 353–362

work page 1983

[8] [8]

Fabio Alexandre Spanhol, Luiz S Oliveira, Caroline Petitjean, and Laurent Heutte

work page