Pith. sign in

REVIEW 4 major objections 6 minor 47 references

A Study of Hybrid and Evolutionary Metaheuristics for Single Hidden Layer Feedforward Neural Network Architecture

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a hybrid PSO-SGD optimizer reduces median training MSE by 90–95 percent compared with ordinary GA and PSO on single-hidden-layer networks for regression tasks, and that crossover-based GA outperforms its…

desk verdict The 90–95% improvement claim is contradicted by the paper's own tables, and the comparison budget is lopsided; underneath there is a modest empirical comparison that would need major rework. read the letter →

arxiv 2506.15737 v1 pith:3FB5AEX5 submitted 2025-06-17 cs.NE cs.AI

Gautam Siddharth Kashyap , Md Tabrez Nafis , Samar Wazir This is my paper · ORCID
classification cs.NEcs.AI
keywords metaheuristicoptimizationparticleswarmgeneticalgorithmPSO-SGDhybridneuralnetworktrainingbuildingblockhypothesissinglehiddenlayerregression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that population-based metaheuristics—particle swarm optimization (PSO), a swarm-search method, and genetic algorithms (GAs), population-based evolutionary search—can train single-hidden-layer feedforward networks as well as or better than stochastic gradient descent (SGD) for small regression networks. Its central claim is that a hybrid PSO-SGD optimizer, which appends a gradient-descent step to the PSO update, cuts median training MSE by 90–95 percent relative to conventional GA and PSO, for example from about 0.02 to about 0.001 on the Sphere function. The authors also find that GA with crossover beats GA without crossover in every configuration, which they interpret as evidence that the Building Block Hypothesis—the idea that favorable partial weight structures are preserved and recombined—applies to neural-network weight training. The study is deliberately limited to regression problems and small architectures, so the claim is about that regime rather than large-scale deep learning.

What carries the argument

The load-bearing mechanism is the hybrid position update in Equation (6): $x_i^{(t+1)} = x_i^{(t)} + \omega v_i^{(t)} + \alpha_1 r_1 \odot (p_i - x_i^{(t)}) + \alpha_2 r_2 \odot (g - x_i^{(t)}) - \eta \nabla_{w_i} E(x_i^{(t)})$. It embeds a deterministic gradient-aligned term into the otherwise stochastic PSO trajectory, so each particle both explores via swarm dynamics and refines via SGD. The learning rate $\eta$ for the gradient term is chosen by a logarithmic grid search over $\{10^{-1}, 10^{-2}, \dots, 10^{-7}\}$, with $\eta = 10^{-5}$ reported as the best setting. This mechanism is what distinguishes the hybrid from plain PSO and explains the reported MSE gains when the termination criterion is iteration count.

What would settle it

Re-run the same experiments with equal total function evaluations for every method—cap population methods at 150,000 evaluations or let SGD run for 3.75 million—and check whether PSO-SGD still beats GA, PSO, and SGD by the reported margin. A second check is whether the GA crossover advantage persists on a classification benchmark such as MNIST rather than only on the regression problems tested here.

Watch

Extended reading notes

Core claim

The central finding is that under iteration-based termination, PSO-SGD is the best or near-best method on small and medium networks, while SGD remains competitive and often better on larger or wider ones. The hybrid update rule adds a gradient descent term to the standard PSO velocity update, letting particles perform local refinement after global exploration. In the authors' measurements, the hybrid reduces median training MSE from around 0.02 to around 0.001 on the Sphere function, a 90–95 percent improvement over conventional GA and PSO; RMHC also reduces MSE by roughly 85–90 percent compared with GA. The paper also reports that on their synthetic benchmarks all non-random methods converge to nearly identical MSE for very deep networks, and that crossover contributes measurable gains to GA.

Load-bearing premise

The headline comparison treats 150 iterations as a fair budget for every method, even though population methods evaluate 25 particles per iteration while SGD evaluates a single solution, so population methods receive 3.75 million function evaluations against SGD's 150,000.

Editorial extensions

If this is right

  • Under iteration-based termination, PSO-SGD and PSO are the strongest methods on small and medium networks, outperforming SGD in several configurations.
  • Under function-evaluation termination, SGD remains the most accurate method on width-scaled problems, so the choice of budget criterion changes the ranking of methods.
  • GA with crossover beats GA without crossover in all tested configurations, implicating recombination as the source of the advantage.
  • RMHC is competitive with population methods on depth-scaled tasks when the budget is function evaluations, but its advantage disappears when iterations are capped.
  • All non-random methods converge to nearly identical MSE for very deep networks, suggesting the practical advantage of metaheuristics is confined to smaller architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reported gains survive equalized evaluation budgets, the natural next test is whether the hybrid's gradient term helps most on non-smooth or deceptive loss surfaces, where PSO alone tends to stall; the authors' CEC functions are all smooth and differentiable, so that mechanism is not directly tested.
  • The GA-with-crossover advantage over GA-without-crossover is indirect evidence for the Building Block Hypothesis; a direct test would track whether specific weight subsets survive recombination and correlate with low error, rather than inferring this from aggregate MSE.
  • The deep-network convergence of all methods to similar MSE suggests the regression targets saturate the capacity of those architectures, which would mean the practical regime for metaheuristic training is exactly the small, structured networks the paper studies, not larger ones.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a hybrid PSO-SGD optimizer for training single-hidden-layer feedforward neural networks on regression tasks, and compares it with GA, GA without crossover, PSO, SGD, RMHC, and RS on three CEC benchmark functions (Rastrigin, Styblinski-Tang, Sphere) and two empirical datasets (CCPP, AFSN). The central claim, stated in the abstract and introduction, is that the hybrid PSO-SGD reduces median training MSE by 90–95% relative to GA and PSO across network sizes, with an example of roughly 0.02 to 0.001 on the Sphere function. The paper also interprets the crossover advantage of GA over GA-nc as evidence for the Building Block Hypothesis in neural network weight space.

Significance. If the central quantitative claim were supported, the paper would provide a practically useful result: a simple hybrid that combines PSO's global search with SGD's local refinement could substantially improve training accuracy for small and medium feedforward networks without GPU-level resources. The manuscript has some strengths: the hybrid update rule is clearly specified in Algorithm 1 and Equation (6), the experimental design includes multiple synthetic functions and two real-world datasets, several baseline optimizers (including RMHC and RS) are included, and the authors report a fixed random seed for reproducibility. However, the main quantitative claim is directly contradicted by the paper's own tables, and the comparison protocol gives population-based methods a much larger function-evaluation budget than SGD. These issues undermine the paper's principal contribution as currently stated.

major comments (4)
  1. [Abstract and Section VI; Table I] The headline claim that PSO-SGD decreases median training MSE by 90–95% relative to GA and PSO is not supported by the reported data. For the Sphere function, Table I shows that for depths 20 and above, all methods except RS reach exactly 0.01823, meaning PSO-SGD yields zero improvement over GA and PSO. For smaller depths, the maximum relative reduction over GA is 89.3% (15-layer row: 0.01755 to 0.00188) and over PSO is 86.2% (10-layer row: 0.01441 to 0.00199); most rows show reductions in the 71–85% range, and several Rastrigin rows show PSO-SGD performing worse than PSO. Additionally, Section VI states that 'minimal distinction was noted between PSO and PSO-SGD across various configurations,' which contradicts the 90–95% improvement claim. The specific example of 0.02 → 0.001 appears nowhere in Table I. Because the central contribution is this quantitative improvement, the claim as written is internally contradicted.
  2. [Section IV.B.3, Termination Criteria] The comparison is undermined by unequal function-evaluation budgets. The paper states that each technique was permitted a maximum of 150 iterations or 3,750,000 function evaluations, and explains that this corresponds to 1,000 samples, 150 iterations, and a population size of 25. This gives population-based methods (GA, PSO, PSO-SGD, and by the same arithmetic RMHC when treated as a population) 25 × 1,000 × 150 = 3,750,000 FEs, whereas SGD evaluates only 1,000 samples per iteration, i.e., 150,000 FEs, 25 times fewer. The statement that direct iteration limitations provide a 'level playing field' is therefore inaccurate. All comparisons in Tables I–IV and Figures 1–4 that are interpreted as algorithmic superiority are confounded by this budget imbalance and do not establish the claimed advantage of population methods over SGD.
  3. [Section III, Equation (6) and learning-rate selection] The learning rate for the gradient term in PSO-SGD, η = 1e-5, is selected by logarithmic grid search over the same benchmark functions (Rastrigin, Styblinski-Tang, Sphere) on which the method is then evaluated. The same holds for the SGD learning rate of 0.1, which is set by 'a parameter scan analogous to that employed in PSO-SGD.' Because the reported results are obtained on the very functions used to tune these hyperparameters, the performance comparisons are subject to tuning circularity: the headline reductions may reflect favorable hyperparameter selection rather than a general property of the hybrid method. The paper provides no separate validation set of functions or cross-function transfer test to show that the chosen settings generalize.
  4. [Sections IV.A and V.A; Tables I–IV] Statistical support is too weak for the quantitative claims. Each configuration is averaged over only four trials, and the paper reports only the median without variance, confidence intervals, or significance tests. Many rows in Tables I and II show identical values across all non-RS methods (e.g., Rastrigin depth 20+ at 0.02333 and Sphere depth 20+ at 0.01823), which suggests a shared convergence floor rather than meaningful method-specific differences. In such settings, differences of order 1e-3 between methods are not interpretable without error bars or repeated-measures comparisons. This is especially problematic for the claim that PSO-SGD consistently outperforms PSO or GA by 90–95%.
minor comments (6)
  1. [Section IV.B.2 and Tables I–IV] The GA without crossover baseline is used throughout the tables and figures but is not described in the methodology; only the main GA is presented in the parameter settings, so it is unclear how the no-crossover variant was generated and whether it otherwise matches GA.
  2. [Table I note] The note says bold values signify the best median training MSE, but the table as reproduced contains no bold entries; the formatting should be corrected or the note removed.
  3. [Section II.B and reference list] The phrase 'Such et al. [22]' is used instead of a proper author name, and reference [22] is an arXiv preprint rather than the published competitive-evolution source; the citation should be corrected and completed.
  4. [Section IV.B.1] The paper says a fixed seed was used with the random number generator but does not report the seed value; reporting the seed would allow the experiments to be reproduced exactly.
  5. [Section IV.A] The relationship between 'ANN layers (Dim)' and the dimensionality m used in Equation (1) is not made explicit; for example, the table lists a 1-layer network with dimension 133, but the derivation of 133 from the architecture is not shown.
  6. [Figures 1–4] The figures are not described in enough detail in the text: axis labels, legend entries, and the definition of 'marker size representing variance' are omitted, making it hard to verify the claims about RMHC and RS performance from the figures.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are empirical benchmark results, and the main tuning concern is a generalization limitation, not a definitional or self-citation reduction.

full rationale

Most of the paper is an empirical benchmark comparison rather than a derivation chain, so there is no step where a claimed prediction is defined from the same fitted input by construction. The one candidate for tuning-related circularity is the PSO-SGD learning rate: Section IV.B.2 states that eta is chosen by logarithmic grid search over {10^-1,...,10^-7} and that eta = 10^-5 was selected based on results 'across all benchmark functions', with SGD's learning rate similarly set to 0.1 'ascertained by a parameter scan analogous to that employed in PSO-SGD hybrid methodologies.' This is hyperparameter selection on the same benchmarks used for evaluation, which is a real limitation for generalization and the authors partly acknowledge it in Section VI.A when they say hyperparameters were optimized for particular configurations and their applicability to other settings is ambiguous. However, this is not circular in the sense required here: the reported median training MSE values are actual optimization outcomes, not quantities forced by the eta choice through an identity or fit, and the paper does not claim to predict held-out performance on unseen functions. No load-bearing self-citation was found: none of the references appear to be authored by the present paper's authors, and no uniqueness theorem or ansatz is imported from prior work by the same group. The discrepancy between the abstract's '90-95%' / '0.02 to 0.001' claim and Table I, where the largest Sphere reductions are about 89% over GA and 86% over PSO with many rows showing no improvement, is an internal-evidence or correctness problem, not circularity. Therefore the honest finding is no significant circularity, score 0.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The paper's central claim rests on hyperparameters (learning rates, mutation scale, population size) that were tuned on the same benchmarks used for evaluation, plus background assumptions about the representativeness of the CEC functions and the validity of training MSE as a proxy for generalization.

free parameters (5)
  • PSO-SGD learning rate eta = 1e-5
    Selected by grid search over {10^-1, ..., 10^-7} with other PSO parameters fixed (Section IV.B.2). Since it is tuned on the same benchmark functions, the reported improvements are conditional on this fitted value.
  • SGD learning rate = 0.1
    The paper states it was 'ascertained by a parameter scan analogous to that employed in PSO-SGD hybrid methodologies' (Section IV.B.2), so it is also fitted to the test tasks.
  • Mutation scale sigma^2 = 0.001
    For RMHC and GA, the paper says 'Various magnitudes were assessed, with sigma^2 = 0.001 demonstrating optimal performance' (Section IV.B.2). This is a hand-chosen parameter that affects the results.
  • Population size = 25
    Chosen following best practices from [46], not fitted to the benchmarks, but it sets the computational budget asymmetry that biases the FE-based comparisons.
  • PSO inertia and acceleration coefficients (omega, alpha1, alpha2)
    The paper says 'keeping all other PSO parameters constant' but does not report their values. These unstated free parameters affect the swarm dynamics and are not available for replication.
assumptions (3)
  • standard math Universal Approximation Theorem (UAT)
    The paper relies on UAT to justify using a single hidden layer feedforward network (Section III, citing [38]). It is a standard theorem, not proven in the paper.
  • domain assumption Training MSE on synthetic functions is a meaningful proxy for optimizer quality
    The paper evaluates optimizers by final training MSE on 1000-point samples of CEC functions and claims this indicates training efficiency and generalization ('initial study indicated minimal variations between training and test MSEs'), without providing test MSE data.
  • domain assumption CEC benchmark functions model relevant ANN loss landscapes
    The paper uses Rastrigin, Styblinski-Tang, and Sphere functions to evaluate ANN training methods, implicitly assuming these capture relevant difficulty characteristics of real neural network training.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Study of Hybrid and Evolutionary Metaheuristics for Single Hidden Layer Feedforward Neural Network Architecture." pith.science (2026). https://pith.science/paper/3FB5AEX5

@misc{pith2026250615737,
  author       = {Pith},
  title        = {Pith review of: A Study of Hybrid and Evolutionary Metaheuristics for Single Hidden Layer Feedforward Neural Network Architecture},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3FB5AEX5}},
  note         = {Machine review of arXiv:2506.15737}
}
read the original abstract

Training Artificial Neural Networks (ANNs) with Stochastic Gradient Descent (SGD) frequently encounters difficulties, including substantial computing expense and the risk of converging to local optima, attributable to its dependence on partial weight gradients. Therefore, this work investigates Particle Swarm Optimization (PSO) and Genetic Algorithms (GAs) - two population-based Metaheuristic Optimizers (MHOs) - as alternatives to SGD to mitigate these constraints. A hybrid PSO-SGD strategy is developed to improve local search efficiency. The findings indicate that the hybrid PSO-SGD technique decreases the median training MSE by 90 to 95 percent relative to conventional GA and PSO across various network sizes (e.g., from around 0.02 to approximately 0.001 in the Sphere function). RMHC attains substantial enhancements, reducing MSE by roughly 85 to 90 percent compared to GA. Simultaneously, RS consistently exhibits errors exceeding 0.3, signifying subpar performance. These findings underscore that hybrid and evolutionary procedures significantly improve training efficiency and accuracy compared to conventional optimization methods and imply that the Building Block Hypothesis (BBH) may still be valid, indicating that advantageous weight structures are retained during evolutionary search.

Figures

Figures reproduced from arXiv: 2506.15737 by the authors.

Figure 1
Figure 1. AFSN dataset: Median MSE performance with FE as the termination [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 3
Figure 3. CCPP dataset: Median MSE performance with FE as the stopping [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 39 canonical work pages

  1. [29]

    A hybrid particle swarm optimization with gradient-based method for training feedforward neural networks,

    J. Zhang, X. Xie, and Y . Zhuang, “A hybrid particle swarm optimization with gradient-based method for training feedforward neural networks,” Neurocomputing, vol. 73, no. 16-18, pp. 3223–3231, 2010. 11

    work page 2010

  2. [1]

    Convolutional neural network classification of cancer cytopathology images: taking breast cancer as an example,

    M. Xiao, Y . Li, X. Yan, M. Gao, and W. Wang, “Convolutional neural network classification of cancer cytopathology images: taking breast cancer as an example,” in Proceedings of the 2024 7th International Conference on Machine Vision and Applications , 2024, pp. 145–149

    work page 2024

  3. [2]

    Goodfellow, Y

    I. Goodfellow, Y . Bengio, A. Courville, and Y . Bengio, Deep learning. MIT press Cambridge, 2016, vol. 1, no. 2

    work page 2016

  4. [3]

    Unsupervised representation learning with deep convolutional generative adversarial networks,

    A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv preprint arXiv:1511.06434, 2015

    arXiv 2015

  5. [4]

    Deep learning,

    Y . LeCun, Y . Bengio, and G. Hinton, “Deep learning,” nature, vol. 521, no. 7553, pp. 436–444, 2015

    2015

  6. [5]

    Dermatologist-level classification of skin cancer with deep neural networks,

    A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun, “Dermatologist-level classification of skin cancer with deep neural networks,” nature, vol. 542, no. 7639, pp. 115–118, 2017

    2017

  7. [6]

    A stochastic approximation method,

    H. Robbins and S. Monro, “A stochastic approximation method,” The annals of mathematical statistics , pp. 400–407, 1951

    1951

  8. [7]

    Identifying and attacking the saddle point problem in high- dimensional non-convex optimization,

    Y . N. Dauphin, R. Pascanu, C. Gulcehre, K. Cho, S. Ganguli, and Y . Bengio, “Identifying and attacking the saddle point problem in high- dimensional non-convex optimization,” Advances in neural information processing systems, vol. 27, 2014

    work page 2014

Show all 47 references
  1. [8]

    A new optimizer using particle swarm theory,

    R. Eberhart and J. Kennedy, “A new optimizer using particle swarm theory,” in Proceedings of the Sixth International Symposium on Micro Machine and Human Science . IEEE, 1995, pp. 39–43

    1995

  2. [9]

    J. H. Holland, Adaptation in Natural and Artificial Systems: An Intro- ductory Analysis with Applications to Biology, Control, and Artificial Intelligence. MIT press, 1992

    1992

  3. [10]

    Yang, Engineering Optimization: An Introduction with Meta- heuristic Applications

    X.-S. Yang, Engineering Optimization: An Introduction with Meta- heuristic Applications. John Wiley & Sons, 2010

    2010

  4. [11]

    Genetic algorithms in control prob- lems,

    D. Dasgupta and Z. Michalewicz, “Genetic algorithms in control prob- lems,” in IEEE International Conference on Evolutionary Computation . IEEE, 1998, pp. 253–258

    1998

  5. [12]

    A hybrid evolutionary algorithm for solving multiobjective optimization problems,

    K. Tan, T. Lee, E. Khor, and Q. Yu, “A hybrid evolutionary algorithm for solving multiobjective optimization problems,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) , vol. 35, no. 2, pp. 195–204, 2005

    2005

  6. [13]

    D. E. Goldberg, Genetic Algorithms in Search, Optimization and Ma- chine Learning. Addison-Wesley, 1989

    1989

  7. [14]

    The particle swarm—explosion, stability, and convergence in a multidimensional complex space,

    M. Clerc and J. Kennedy, “The particle swarm—explosion, stability, and convergence in a multidimensional complex space,” IEEE Transactions on Evolutionary Computation , vol. 6, no. 1, pp. 58–73, 2002

    2002

  8. [15]

    Optimal power flow using particle swarm optimization,

    M. Abido, “Optimal power flow using particle swarm optimization,” International Journal of Electrical Power & Energy Systems , vol. 24, no. 7, pp. 563–571, 2002

    2002

  9. [16]

    Task scheduling based on particle swarm optimization in grid environment,

    L. Zhang, Y . Wu, H. Liu, and X. Yuan, “Task scheduling based on particle swarm optimization in grid environment,” IEEE International Conference on Services Computing (SCC 2007) , pp. 235–242, 2007

    2007

  10. [17]

    Human-level control through deep reinforce- ment learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. Rusu, J. Veness, M. Bellemare, A. Graves, M. Riedmiller, A. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep reinforc...

    2015

  11. [18]

    Policy gradient methods for reinforcement learning with function approximation,

    R. Sutton, D. McAllester, S. Singh, and Y . Mansour, “Policy gradient methods for reinforcement learning with function approximation,” in Advances in Neural Information Processing Systems , 2000, pp. 1057– 1063

    2000

  12. [19]

    Increasing generality in machine learning through neuroevolution,

    S. Risi and J. Togelius, “Increasing generality in machine learning through neuroevolution,” Nature Machine Intelligence, vol. 2, no. 8, pp. 428–436, 2020

    2020

  13. [20]

    Evolving neural networks through augmenting topologies,

    K. Stanley and R. Miikkulainen, “Evolving neural networks through augmenting topologies,” Evolutionary Computation, vol. 10, no. 2, pp. 99–127, 2002

    2002

  14. [21]

    Eiben and J

    A. Eiben and J. Smith, Introduction to Evolutionary Computing . Springer, 2003

    2003

  15. [22]

    Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning,

    F. Such, V . Madhavan, E. Conti, J. Lehman, K. Stanley, and J. Clune, “Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning,” arXiv preprint arXiv:1712.06567, 2017

    2017 arXiv

  16. [23]

    Evolution strategies as a scalable alternative to reinforcement learning,

    T. Salimans, J. Ho, X. Chen, and I. Sutskever, “Evolution strategies as a scalable alternative to reinforcement learning,” arXiv preprint arXiv:1703.03864, 2017

    2017 arXiv

  17. [24]

    Mitchell, An Introduction to Genetic Algorithms

    M. Mitchell, An Introduction to Genetic Algorithms . MIT Press, 1998

    1998

  18. [25]

    Adaptive selection methods for genetic algorithms,

    J. Baker, “Adaptive selection methods for genetic algorithms,” in Pro- ceedings of the 1st International Conference on Genetic Algorithms , 1985, pp. 101–111

    1985

  19. [26]

    A comparative evaluation of genetic algorithms and backpropagation for training neural networks,

    N. Gupta and T. Sexton, “A comparative evaluation of genetic algorithms and backpropagation for training neural networks,” Neurocomputing, vol. 38, pp. 91–105, 2001

    2001

  20. [27]

    Efficient and accurate parallel genetic algorithms,

    E. Cant ´u-Paz, “Efficient and accurate parallel genetic algorithms,” Kluwer Academic Publishers , 2003

    2003

  21. [28]

    Particle swarm optimization: Techniques, system and challenges,

    U. Ojha, A. Abraham, and V . Sn ´aˇsel, “Particle swarm optimization: Techniques, system and challenges,” Swarm and Evolutionary Compu- tation, vol. 39, pp. 1–24, 2017

    2017

  22. [30]

    Particle swarm optimization: Velocity initialization and parameters selection,

    A. Erskine, A. P. Engelbrecht, A. Salman, and I. Cloete, “Particle swarm optimization: Velocity initialization and parameters selection,” Applied Soft Computing, vol. 24, pp. 1033–1043, 2014

    2014

  23. [31]

    Particle swarm optimization for product unit neural networks,

    M. Ismail and A. Engelbrecht, “Particle swarm optimization for product unit neural networks,” Neurocomputing, vol. 174, pp. 1089–1102, 2016

    2016

  24. [32]

    Leapfrog optimizer: A new method for optimizing neural networks,

    A. Engelbrecht, “Leapfrog optimizer: A new method for optimizing neural networks,” Journal of Applied Mathematics , vol. 2014, pp. 1– 13, 2014

    2014

  25. [33]

    Performance comparison of pso, ga, and sgd algorithms on different function evaluation criteria,

    S. Khan and A. Sahai, “Performance comparison of pso, ga, and sgd algorithms on different function evaluation criteria,” International Journal of Computational Intelligence Systems , vol. 12, no. 1, pp. 167– 177, 2019

    2019

  26. [34]

    A comparative study of genetic algorithms for neural network training,

    H. S. Lopes and A. F. de Carvalho, “A comparative study of genetic algorithms for neural network training,” in Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics , vol. 4, 2000, pp. 3241–3246

    2000

  27. [35]

    Parameter selection in particle swarm op- timization,

    Y . Shi and R. Eberhart, “Parameter selection in particle swarm op- timization,” in Proceedings of the 7th International Conference on Evolutionary Programming. Springer, 1998, pp. 591–600

    1998

  28. [36]

    Benchmarking evolutionary algorithms for training neural networks,

    C. H. L. Bezerra, L. N. de Castro, and C. G. Lopes, “Benchmarking evolutionary algorithms for training neural networks,” in 2010 IEEE Congress on Evolutionary Computation (CEC) , 2010, pp. 2588–2595

    2010

  29. [37]

    Combining metaheuristics and gradient-based local search for training neural networks,

    E. Alba, B. Dorronsoro, and A. N. Z. Ulate, “Combining metaheuristics and gradient-based local search for training neural networks,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) , vol. 38, no. 2, pp. 381–403, 2008

    2008

  30. [38]

    A survey on universal approximation theorems,

    M. T. Augustine, “A survey on universal approximation theorems,” arXiv preprint arXiv:2407.12895, 2024

    2024 arXiv

  31. [39]

    Mitchell, An Introduction to Genetic Algorithms

    M. Mitchell, An Introduction to Genetic Algorithms . MIT Press, 1996

    1996

  32. [40]

    Random search for hyper-parameter opti- mization,

    J. Bergstra and Y . Bengio, “Random search for hyper-parameter opti- mization,” in Journal of Machine Learning Research , vol. 13, no. 2, 2012, pp. 281–305

    2012

  33. [41]

    Performance analysis of evolutionary algorithms on rastrigin function,

    M. A. Momin and M. A. Matin, “Performance analysis of evolutionary algorithms on rastrigin function,” in 2nd International Conference on Electrical and Computer Engineering , 2005, pp. 183–186

    2005

  34. [42]

    A literature survey of benchmark functions for global optimization problems,

    M. Jamil and X.-S. Yang, “A literature survey of benchmark functions for global optimization problems,” International Journal of Mathematical Modelling and Numerical Optimisation, vol. 4, no. 2, pp. 150–194, 2013

    2013

  35. [43]

    Problem definitions and evaluation criteria for the cec 2013 special session on real-parameter optimization,

    J. J. Liang, B. Y . Qu, and P. N. Suganthan, “Problem definitions and evaluation criteria for the cec 2013 special session on real-parameter optimization,” in Technical Report 201212, Computational Intelligence Laboratory, Zhengzhou University, 2013

    2013

  36. [44]

    Prediction of full load electrical power output of a base load operated combined cycle power plant using machine learning methods,

    P. Tufekci, “Prediction of full load electrical power output of a base load operated combined cycle power plant using machine learning methods,” International Journal of Electrical Power & Energy Systems , vol. 60, pp. 126–140, 2014

    2014

  37. [45]

    An empirical evaluation of ensemble-based multi-objective optimization algorithms: the case of airfoil self-noise,

    J. C. Duarte, A. G. A. M. de Souza, and A. C. G. Melo, “An empirical evaluation of ensemble-based multi-objective optimization algorithms: the case of airfoil self-noise,” Applied Soft Computing, vol. 30, pp. 166– 182, 2015

    2015

  38. [46]

    Best practices for comparing optimization algorithms,

    V . Beiranvand, W. Hare, and Y . Lucet, “Best practices for comparing optimization algorithms,” Optimization and Engineering , vol. 18, pp. 815–848, 2017

    2017

  39. [47]

    A survey on activation functions and their relation with xavier and he normal initialization,

    L. Datta, “A survey on activation functions and their relation with xavier and he normal initialization,” arXiv preprint arXiv:2004.06632 , 2020

    2004 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.