Pith. sign in

REVIEW 5 major objections 4 minor 61 references

Foxtsage vs. Adam: Revolution or Evolution in Optimization?

T0 review · 5 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that Foxtsage, a FOX-TSA-guided SGD, reduces training loss by 42.03% versus Adam across three benchmark datasets, at 4.3x the compute time.

desk verdict A population-based learning-rate search wrapped around SGD, but the comparison with Adam is invalidated by a confounded evaluation protocol and unreported compute budgets. read the letter →

arxiv 2412.17855 v1 pith:DOFSXXGE submitted 2024-12-20 cs.NE cs.LG

Sirwan A. Aula , Tarik A. Rashid This is my paper · ORCID
classification cs.NEcs.LG
keywords FoxtsageFOX-TSAStochasticGradientDescentAdamoptimizerlearningrateadaptationneuralnetworkoptimizationhybridmetaheuristicbenchmarkcomparison
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a hybrid optimizer called Foxtsage—SGD with its learning rate steered by the FOX-TSA population search—trains neural networks better than the widely used Adam optimizer. On MNIST, CIFAR-10, and IMDB, across logistic regression, MLP, and CNN models, the authors report a 42.03% lower mean training loss, a 42.19% lower loss standard deviation, and modest gains in accuracy, precision, recall, and F1-score, at a 330.87% higher mean computation time. If true, the result would show that a population-based learning-rate controller can beat Adam's adaptive moments on small-scale networks, though only when wall-clock time is not the limiting factor. The claim rests on the Section 3.2.3 pseudocode, in which each candidate learning rate is evaluated by one epoch of SGD on a single shared model before FOX-TSA updates the population around the best loss.

What carries the argument

The central mechanism is the Foxtsage optimizer, a population-based learning-rate controller wrapped around SGD. A population of candidate learning rates is initialized uniformly in $[\eta_{\min}, \eta_{\max}]$; each candidate trains the model for one epoch via $\theta \leftarrow \theta - \mathrm{candidate\_lr} \times \nabla L(\theta)$, the resulting loss is compared to the current best, and FOX-TSA then perturbs the population around the best learning rate using an exploration move (multiply by a Gaussian perturbation when $r<0.5$) or an exploitation move (divide by a Gaussian perturbation when $r\ge 0.5$), clipping back into bounds. The load-bearing identity is Eq. (6), $\eta_t = \eta_{\mathrm{base}}/(1 + \alpha f_{\mathrm{best}}(t))$, which ties the SGD step size to the best fitness found by FOX-TSA; the pseudocode evaluates candidates sequentially on one shared model, so the 'best' learning rate is selected on a trajectory that already contains earlier candidates' updates.

What would settle it

Rerun the Foxtsage pseudocode with the model parameters reset to a common random initialization before each candidate learning rate is evaluated, or with each candidate training a fresh copy of the model; if the 42% loss advantage over Adam disappears, the claimed benefit is an artifact of the sequential update rather than of learning-rate search.

Watch

Extended reading notes

Core claim

Foxtsage is presented as an adaptive learning-rate design: at each iteration, the hybrid FOX-TSA algorithm searches a population of candidate learning rates, and the best one found so far sets the SGD step via $\eta_t = \eta_{\mathrm{base}}/(1 + \alpha f_{\mathrm{best}}(t))$. The paper's headline result, aggregated in its Table 16, is that Foxtsage reduces mean training loss by 42.03% (9.508 vs. 16.402) and loss standard deviation by 42.19% (20.86 vs. 36.085) relative to Adam, with accuracy mean up 0.78%, precision up 0.91%, recall up 1.02%, and F1-score up 0.89%. The same table reports a 330.87% increase in mean computation time, which the authors acknowledge as the main trade-off. The authors interpret the lower loss and smaller variance as evidence that Foxtsage is a robust alternative to Adam for neural-network training, and they argue that the FOX-TSA exploration/exploitation balance is what lets SGD escape the pitfalls of a static learning rate.

Load-bearing premise

The load-bearing premise is that each candidate learning rate is evaluated fairly, but the pseudocode updates a single shared model sequentially for all candidates, so later candidates inherit parameter changes from earlier ones and the reported 'best' learning rate may simply be the last one tried.

Editorial extensions

If this is right

  • Foxtsage reduces mean training loss by 42.03% relative to Adam across the benchmark settings, with a 42.19% reduction in loss standard deviation.
  • Accuracy, precision, recall, and F1-score improve on average by 0.78%, 0.91%, 1.02%, and 0.89%, respectively.
  • The loss improvements grow with the computational budget: Setting 2 (50 iterations, population 30) yields larger reductions than Setting 1 (5 iterations, population 10).
  • These gains come at a 330.87% increase in mean computation time and a 266.13% increase in its standard deviation, so Foxtsage is not a drop-in replacement under tight time budgets.
  • The authors report statistically significant differences between the two settings (p = 0.0137 for loss mean, p = 0.0371 for accuracy mean, p = 0.0488 for F1-score) and interpret Foxtsage as a robust alternative to Adam.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If Foxtsage's loss gains largely reflect the extra gradient evaluations spent evaluating a population, then a fairer baseline is Adam run for the same number of epochs or the same wall-clock time; the paper's own 330% time overhead makes this a live possibility.
  • The sequential update in Section 3.2.3 effectively turns the 'population' into a single trajectory with a time-varying learning rate; testing Foxtsage with population size 1 would reveal how much of the benefit comes from FOX-TSA search versus an implicit learning-rate schedule.
  • A direct comparison to Adam with cosine annealing or other standard learning-rate schedules would clarify whether Foxtsage's advantage is specific to the FOX-TSA mechanism or just a form of adaptive decay.
  • On CIFAR-10 Setting 2, Foxtsage's accuracy mean (0.7623) is slightly below Adam's (0.7681) despite a lower loss mean, so the claimed superiority is loss-driven; practitioners should check whether the loss metric aligns with their deployment metric before adopting Foxtsage.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes Foxtsage, a hybrid optimizer that wraps Stochastic Gradient Descent with a population-based learning-rate controller derived from the FOX-TSA metaheuristic, and benchmarks it against Adam on MNIST, CIFAR-10, and IMDB using MLPs, CNNs, and logistic regression. The central claim is that Foxtsage reduces mean loss by 42.03% and loss standard deviation by 42.19% relative to Adam, with modest accuracy/F1 gains, at roughly 3.3x higher compute cost, based on results aggregated over two settings (5 iterations with population 10, and 50 iterations with population 30).

Significance. If the central claim were sound, a population-based controller that dynamically schedules the learning rate for SGD and beats Adam on standard benchmarks would be a noteworthy empirical contribution to the optimizer literature. The paper has some strengths: it compares across multiple datasets and architectures, reports variability metrics (standard deviations), and is transparent about the large computational overhead. However, the experimental protocol as described does not support the headline claim: the algorithm's candidate evaluation step is confounded, the compute budget is not matched, and the aggregate Table 16 mixes losses of completely different scales. These issues are load-bearing rather than cosmetic, so the contribution as stated is not established.

major comments (5)
  1. [Section 3.2.3, pseudocode] The pseudocode's candidate evaluation is invalid. In the inner loop, for each candidate_lr, the same shared parameter vector θ is advanced with SGD for one epoch (θ ← θ - candidate_lr × ∇L(θ)), and then current_loss is computed on that already-updated θ. There is no re-initialization or per-candidate reset, so the loss attributed to candidate j already includes the cumulative updates from candidates 1 through j-1 in the same iteration and from all previous iterations. Consequently, the comparison that updates best_lr/best_loss does not isolate the effect of any single candidate learning rate. The prose summary ('For each learning rate in the population, the model is trained for one epoch using SGD') is ambiguous: if the implementation actually trains a fresh model per candidate, the pseudocode is misleading; if it follows the pseudocode, the selection of the best learning rate is confounded. Either way, the reported gains over Adam cannot be attributed to the FOX-TSA search mechanism as described.
  2. [Section 3.4.2 and Section 4.1] The comparison with Adam is not compute-matched. Foxtsage is run with 5 iterations × 10 candidates and 50 iterations × 30 candidates, which translates to 50 and 1500 SGD epoch-equivalents respectively, yet the number of epochs used for Adam is never reported. Loss values such as 99.45 (Table 5) and 231.36 (Table 9) for Foxtsage versus much lower values in other rows suggest that the two optimizers are not being compared after a comparable number of gradient steps. Without matching the number of epochs, iterations, or wall-clock budget, the claimed 42.03% loss reduction is not a meaningful comparison of optimizer quality.
  3. [Table 16 and Eq. (7)] The aggregate loss metric in Table 16 is statistically meaningless because it averages losses across datasets that use different loss scales and different loss functions. For example, MNIST logistic regression reports loss means around 0.28-0.30, MNIST MLP reports 46.82-99.45, and CIFAR-10 CNN reports 0.12-231.36. Eq. (7) defines binary cross-entropy for binary classification, but the paper applies it to multi-class MNIST and CIFAR-10, and the reported values are far larger than a per-sample cross-entropy would be, suggesting either summed losses or an unnormalized objective. Averaging these heterogeneous quantities into a single 'Loss Mean' (16.402 vs 9.508) and computing a 42.03% improvement does not support the abstract's central quantitative claim.
  4. [Table 13 and Section 4.2.2] Table 13 contains duplicated and likely incorrect entries: the rows for 'CNN (CIFAR-10)' and 'Logistic Regression (MNIST)' both report Loss Mean Improvement 41.34572907 and Accuracy Mean Improvement 0.747593951 in Setting 2, while the corresponding loss values in Table 14 (0.120085 vs 0.204734) yield a different percentage change. This inconsistency undermines the reliability of the reported percentage improvements and needs to be corrected and re-verified.
  5. [Table 15 and Section 4.2.3] The statistical significance reporting is not coherent. Table 15 is titled as comparing Setting 1 and Setting 2 ('between Setting 1 and Setting 2'), yet the text in Section 4.2.3 states that the p-values confirm that 'the performance differences between Foxtsage and Adam were significant.' No test name, sample size, or pairing structure is given, and the 'Statistic' column contains unexplained integers 4, 7, and 8. As written, the statistical tests do not support either claim reliably.
minor comments (4)
  1. [General] The paper contains numerous typos and grammatical errors, including 'optimisation' vs 'optimization' inconsistent usage, 'Hof Fman' in references, and the conclusion's 'precision 1.91' which should be '0.91%'.
  2. [Figures 15-16] Figure captions are incorrectly labeled: Figure 15 says 'Figure 13: CNN Training Loss on MNIST for Setting 2' and Figure 16 says 'Figure 14: CNN Validation Accuracy on MNIST for Setting 2', duplicating the earlier figure numbers.
  3. [Section 4.2.4] The sentence 'Precision improved by 0.91% and recall by 1.02%, with their standard deviations improving by 7.95% and 7.87%' is unclear whether the standard deviation values are relative improvements or absolute changes.
  4. [References] Several references lack complete bibliographic information, such as page numbers or venue details (e.g., reference 26 for Kingma and Ba, reference 9 for Defossez et al.), and some citations are repeated with inconsistent author names (e.g., 'Aula, S.A.' vs 'Sirwan A. Aula').

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: Foxtsage is an externally benchmarked empirical method, and the self-citations to prior FOX-TSA work are motivational rather than load-bearing.

full rationale

The paper's central claim is an empirical benchmark result: Foxtsage reduces loss mean by 42.03% and loss standard deviation by 42.19% relative to Adam on MNIST, CIFAR-10, and IMDB (Abstract and Table 16). Those numbers come from the experiments in Section 4, not from the defining equations. The only self-referential element is Eq. 6, where the learning rate schedule is η_t = η_base/(1 + α·f_best(t)), with f_best(t) being the best fitness or loss of the model being trained. That is a feedback control law, not a derivation: the schedule could perform badly, and the paper tests it against an external optimizer and external datasets. The FOX-TSA mechanism is motivated by the authors' prior work, but the reported comparison does not reduce to those citations; the benchmark data and the Adam baseline are outside the fitted values. The pseudocode in Section 3.2.3 is a serious experimental-validity concern because the shared model θ is updated sequentially by every candidate learning rate, so current_loss for a later candidate reflects updates made by earlier candidates in the same and previous iterations. That makes the selection of best_lr a confounded evaluation of learning-rate quality, but it is a validity flaw rather than circularity: no quantity is defined as, or predicted from, the very quantity it is used to explain. The statistical reporting is also unreliable, but unreliability is not circularity. Under the required standard of exhibiting a specific Eq.-to-Eq. or fitted-parameter-to-prediction reduction, no load-bearing circular step can be identified.

Assumptions & free parameters 6 free parameters · 4 assumptions · 1 invented entities

The central claim rests on several parameters chosen by hand (eta_base, alpha, eta_min, eta_max, population size, iteration count, FOX-TSA weights) and on at least one questionable domain assumption: that averaging losses across datasets of different scales is meaningful, and that a shared-parameter candidate evaluation in the pseudocode yields unbiased comparisons. No new physical or mathematical entities are introduced beyond the Foxtsage algorithm itself.

free parameters (6)
  • eta_base = 0.01
    Initial SGD learning rate, set by hand in Section 3.4.2.
  • alpha = not reported
    Decay control parameter in Eq. 6; value never specified, affects the learning-rate schedule.
  • eta_min, eta_max = not reported
    Bounds for the learning-rate population in the pseudocode (Section 3.2.3); values affect exploration and exploitation.
  • Population size P = 10 or 30
    Two settings in Section 3.4.2; chosen by hand without justification.
  • Number of iterations I = 5 or 50
    Two settings in Section 3.4.2; chosen by hand without justification.
  • FOX-TSA exploration/exploitation weights = not reported
    Mentioned in Section 3.4.2 as hyperparameters but no values are given.
assumptions (4)
  • standard math Adam's update equations (1)-(4) are a correct and appropriate baseline optimizer.
    The paper relies on the standard Adam formulation for the comparison baseline.
  • domain assumption The loss function in Eq. 7, stated as binary cross-entropy, applies to all datasets including the 10-class MNIST and CIFAR-10 tasks.
    Section 3.4.3 introduces Eq. 7 as binary cross-entropy, but the experiments include multi-class classification, where categorical cross-entropy would be required; no modification is described.
  • domain assumption Averaging per-dataset loss means across datasets with different loss scales yields a meaningful aggregate loss mean.
    Table 16 and the abstract compute a single 42.03% reduction by averaging losses whose values range from 0.27 to 156.57; this assumes scale-invariant comparability.
  • ad hoc to paper Each candidate learning rate in the population can be evaluated independently by training a shared parameter vector theta for one epoch and reading the loss.
    This is the load-bearing evaluation assumption in the pseudocode of Section 3.2.3; the text never resets theta between candidates, so the losses are not independent.
invented entities (1)
  • Foxtsage optimizer
    purpose: A hybrid algorithm that wraps FOX-TSA population search around SGD to adapt the learning rate.
    Foxtsage is introduced and evaluated only in this paper; no falsifiable handle outside the reported benchmarks is given, and no external validation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Foxtsage vs. Adam: Revolution or Evolution in Optimization?." pith.science (2026). https://pith.science/paper/DOFSXXGE

@misc{pith2026241217855,
  author       = {Pith},
  title        = {Pith review of: Foxtsage vs. Adam: Revolution or Evolution in Optimization?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DOFSXXGE}},
  note         = {Machine review of arXiv:2412.17855}
}
read the original abstract

Optimization techniques are pivotal in neural network training, shaping both predictive performance and convergence efficiency. This study introduces Foxtsage, a novel hybrid optimisation approach that integrates the Hybrid FOX-TSA with Stochastic Gradient Descent for training Multi-Layer Perceptron models. The proposed Foxtsage method is benchmarked against the widely adopted Adam optimizer across multiple standard datasets, focusing on key performance metrics such as training loss, accuracy, precision, recall, F1-score, and computational time. Experimental results demonstrate that Foxtsage achieves a 42.03% reduction in loss mean (Foxtsage: 9.508, Adam: 16.402) and a 42.19% improvement in loss standard deviation (Foxtsage: 20.86, Adam: 36.085), reflecting enhanced consistency and robustness. Modest improvements in accuracy mean (0.78%), precision mean (0.91%), recall mean (1.02%), and F1-score mean (0.89%) further underscore its predictive performance. However, these gains are accompanied by an increased computational cost, with a 330.87% rise in time mean (Foxtsage: 39.541 seconds, Adam: 9.177 seconds). By effectively combining the global search capabilities of FOX-TSA with the stability and adaptability of SGD, Foxtsage presents itself as a robust and viable alternative for neural network optimization tasks.

Figures

Figures reproduced from arXiv: 2412.17855 by the authors.

Figure 13
Figure 13. CNN Training Loss on MNIST for Setting [PITH_FULL_IMAGE:figures/full_fig_p023_13.png] view at source ↗
Figure 14
Figure 14. CNN Validation Accuracy on MNIST for Setting [PITH_FULL_IMAGE:figures/full_fig_p023_14.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 41 canonical work pages

  1. [1]

    Fitness Dependent Optimizer: Inspired by the Bee Swarming Reproductive Process

    Abdullah, Jaza Mahmood, and Tarik Ahmed. 2019. “Fitness Dependent Optimizer: Inspired by the Bee Swarming Reproductive Process.” IEEE Access 7:43473–86. https://doi.org/10.1109/ACCESS.2019.2907012

    arXiv 2019

  2. [2]

    Mode Choice Behavior of High School Goers: Evaluating Logistic Regression and MLP Neural Networks

    Assi, Khaled J., Kh Md Nahiduzzaman, Nedal T. Ratrout, and Adel S. Aldosary. 2018. “Mode Choice Behavior of High School Goers: Evaluating Logistic Regression and MLP Neural Networks.” Case Studies on Transport Policy 6 (2). https://doi.org/10.1016/j.cstp.2018.04.006

    work page doi:10.1016/j.cstp.2018.04.006 2018

  3. [3]

    FOX -TSA: Navigating Complex Search Spaces and Superior Performance in Benchmark and Real -World Optimization Problems

    Aula, S.A., and T.A. Rashid. 2024. “FOX -TSA: Navigating Complex Search Spaces and Superior Performance in Benchmark and Real -World Optimization Problems.” Ain Shams Engineering Journal . https://doi.org/10.1016/j.asej.2024.103185

    arXiv 2024

  4. [4]

    FOX -TSA Hybrid Algorithm: Advancing for Superior Predictive Accuracy in Tourism-Driven Multi-Layer Perceptron Models

    Aula, Sirwan A., and Tarik A. Rashid. 2024a. “FOX -TSA Hybrid Algorithm: Advancing for Superior Predictive Accuracy in Tourism-Driven Multi-Layer Perceptron Models.” Systems and Soft Computing, December, 200178. https://doi.org/10.1016/J.SASC.2024.200178

    work page arXiv

  5. [5]

    A Novel Adaptive Momentum Method for Medical Image Classification Using Convolutional Neural Network

    Aytaç, Utku Can, Ali Güneş, and Naim Ajlouni. 2022. “A Novel Adaptive Momentum Method for Medical Image Classification Using Convolutional Neural Network.” BMC Medical Imaging 22 (1). https://doi.org/10.1186/s12880-022-00755-z

    work page doi:10.1186/s12880-022-00755-z 2022

  6. [6]

    Towards the AlexNet Moment for Homomorphic Encryption: HCNN, the First Homomorphic CNN on Encrypt ed Data with GPUs

    Badawi, Ahmad Al, Chao Jin, Jie Lin, Chan Fook Mun, Sim Jun Jie, Benjamin Hong Meng Tan, Xiao Nan, Khin Mi Mi Aung, and Vijay Ramaseshan Chandrasekhar. 2021. “Towards the AlexNet Moment for Homomorphic Encryption: HCNN, the First Homomorphic CNN on Encrypt ed Data with GPUs.” IEEE Transactions on Emerging Topics in Computing 9 (3). https://doi.org/10.1109...

    arXiv 2021

  7. [7]

    Lessons from the Evolutionary Computation Bestiary

    Campelo, Felipe, and Claus Aranha. 2023. “Lessons from the Evolutionary Computation Bestiary.” Artificial Life 29 (4). https://doi.org/10.1162/artl_a_00402

    doi:10.1162/artl_a_00402 2023

  8. [8]

    Fish Detection Using Deep Learning

    Cui, Suxia, Yu Zhou, Yonghui Wang, and Lujun Zhai . 2020. “Fish Detection Using Deep Learning.” Applied Computational Intelligence and Soft Computing 2020. https://doi.org/10.1155/2020/3738108. 39

    work page doi:10.1155/2020/3738108 2020

Show all 61 references
  1. [9]

    On the Convergence of Adam and Adagrad

    Defossez, Alexandre, Leon Bottou, Francis Bach, and Nicolas Usunier. 2020. “On the Convergence of Adam and Adagrad.” ArXiv

    2020

  2. [10]

    Comparative Analysis of Optimizers in Deep Neural Networks

    Desai, Chitra. 2020. “Comparative Analysis of Optimizers in Deep Neural Networks.” International Journal of Innovative Science and Research Technology 5 (10)

    2020

  3. [11]

    Convergence Guarantees for RMSProp and ADAM in Non-Convex Optimization and an Empirical Comparison to Nesterov Acceleration

    De, Soham, Anirbit Mukherjee, and Enayat Ullah. 2018. “Convergence Guarantees for RMSProp and ADAM in Non-Convex Optimization and an Empirical Comparison to Nesterov Acceleration.” ArXiv: Learning . https://api.semanticscholar.org/CorpusID:52944421

    2018

  4. [12]

    Intelligent Computer -Aided Model for Efficient Diagnosis of Digital Breast Tomosynthesis 3D Imaging Using Deep Learning

    El-Shazli, Alaa M.Adel, Sherin M. Youssef, and Abdel Hamid Soliman. 2022. “Intelligent Computer -Aided Model for Efficient Diagnosis of Digital Breast Tomosynthesis 3D Imaging Using Deep Learning.” Applied Sciences (Switzerland) 12 (11). https://doi.org/10.3390/app12115736

    2022 doi

  5. [13]

    Multimodal Multi-Objective Football Game Algorithm for Optimizing Test Task Scheduling Problems

    Fadakar, Elyas. 2023. “Multimodal Multi-Objective Football Game Algorithm for Optimizing Test Task Scheduling Problems.” In 2023 IEEE Symposium Series on Computational Intelligence, SSCI 2023 . https://doi.org/10.1109/SSCI52147.2023.10371976

    2023

  6. [14]

    A Hybrid Method for Recommendation Systems Based on Tourism with an Evolutionary Algorithm and Topsis Model

    Forouzandeh, Saman, Mehrdad Rostami, and Kamal Berahmand. 2022. “A Hybrid Method for Recommendation Systems Based on Tourism with an Evolutionary Algorithm and Topsis Model.” Fuzzy Information and Engineering 14 (1). https://doi.org/10.1080/16168658.2021.2019430

    2022

  7. [15]

    Advances in Tree Seed Algorithm: A Comprehensive Survey

    Gharehchopogh, Farhad Soleimanian. 2022. “Advances in Tree Seed Algorithm: A Comprehensive Survey.” Archives of Computational Methods in Engineering 29 (5): 3281 –3304. https://doi.org/10.1007/s11831 -021- 09698-0

    2022 doi

  8. [16]

    Cross -Domain Object Detection Based Foggy Image Enhancement

    Guo, Qiang, Shiliang Pu, Shifeng Zhang, and Bo Li. 2022. “Cross -Domain Object Detection Based Foggy Image Enhancement.” Journal of Image and Graphics 27 (5). https://doi.org/10.11834/jig.210994

    2022 doi

  9. [17]

    An Efficient Optimization Approach for Designing Machine Learning Models Based on Genetic Algorithm

    Hamdia, Khader M., Xiaoying Zhuang, and Timon Rabczuk. 2021. “An Efficient Optimization Approach for Designing Machine Learning Models Based on Genetic Algorithm.” Neural Computing and Applications 33 (6). https://doi.org/10.1007/s00521-020-05035-x

    2021 doi

  10. [18]

    A Pilot Study on the Feasibility of Using P -Plots for Signal Detection in Pharmacovigilance

    Hauben, Manfred, Chen Zou, Ed Whalen, Wei Wang, and Li Hua Zhang. 2015. “A Pilot Study on the Feasibility of Using P -Plots for Signal Detection in Pharmacovigilance.” Statistics in Biopharmaceutical Research 7 (1). https://doi.org/10.1080/19466315.2014.1002628

    2015

  11. [19]

    Ensemble Learning in CNN Augmented with Fully Connected Subnetworks

    Hirata, Daiki, and Norikazu Takahashi. 2023. “Ensemble Learning in CNN Augmented with Fully Connected Subnetworks.” IEICE Transactions on Information and Systems E106.D (7). https://doi.org/10.1587/transinf.2022EDL8098

    2023 doi

  12. [20]

    The Usefulness of Socio-Demographic Variables in Predicting Purchase Decisions: Evidence from Machine Learning Procedures

    Islam, Towhidul, Nigel Meade, Richard T. Carson, Jordan J. Louviere, and Juan Wang. 2022. “The Usefulness of Socio-Demographic Variables in Predicting Purchase Decisions: Evidence from Machine Learning Procedures.” Journal of Business Research 151. https://doi.org/10.1016/j.jb...

    2022 doi

  13. [21]

    A Review on Genetic Algorithm: Past, Present, and Future

    Katoch, Sourabh, Sumit Singh Chauhan, and Vijay Kumar. 2021. “A Review on Genetic Algorithm: Past, Present, and Future.” Multimedia Tools and Applications 80 (5). https://doi.org/10.1007/s11042-020-10139-6

    2021 doi

  14. [22]

    Chaotic Whale Optimization Algorithm

    Kaur, Gaganpreet, and Sankalap Arora. 2018. “Chaotic Whale Optimization Algorithm.” Journal of Computational Design and Engineering 5 (3): 275–84. https://doi.org/10.1016/j.jcde.2017.12.006

    2018 doi

  15. [23]

    Particle Swarm Optimization

    Kennedy, James, Russell Eberhart, and bls gov. 1995. “Particle Swarm Optimization.” https://doi.org/10.1109/ICNN.1995.488968

    1995

  16. [24]

    A Comparative Study of Series Arima/Mlp Hybrid Models for Stock Price Forecasting

    Khashei, Mehdi, and Zahra Hajirahimi. 2019. “A Comparative Study of Series Arima/Mlp Hybrid Models for Stock Price Forecasting.” Communications in Statistics: Simulation and Computation 48 (9). https://doi.org/10.1080/03610918.2018.1458138

    2019

  17. [25]

    CBR Prediction of Pavement Materials in Unsoaked Condition Using LSSVM, LSTM -RNN, and ANN Approaches

    Khatti, Jitendra, and Kamaldeep Singh Grover. 2024. “CBR Prediction of Pavement Materials in Unsoaked Condition Using LSSVM, LSTM -RNN, and ANN Approaches.” International Journal of Pavement Research and Technology 17 (3). https://doi.org/10.1007/s42947-022-00268-6

    2024 doi

  18. [26]

    Adam: A Method for Stochastic Optimization,

    Kingma, Diederik P., and Jimmy Ba. 2014a. “Adam: A Method for Stochastic Optimization,” December. http://arxiv.org/abs/1412.6980

    arXiv

  19. [27]

    TSA: Tree -Seed Algorithm for Continuous Optimization

    Kiran, Mustafa Servet. 2015. “TSA: Tree -Seed Algorithm for Continuous Optimization.” Expert Systems with Applications 42 (19): 6686–98. https://doi.org/10.1016/j.eswa.2015.04.055

    2015 doi

  20. [28]

    Power Line Recognition Method via Fully Convolutional Network

    Liu, Jiawei, Yuanxiang Li, Zheng Gong, Xingang Liu, and Yongjun Zhou. 2020. “Power Line Recognition Method via Fully Convolutional Network.” Journal of Image and Graphics 25 (5). https://doi.org/10.11834/jig.190316. 40

    2020 doi

  21. [29]

    Learning Rate Optimization in Convolutional Neural Networks for Medical Images Classification

    Li, Yuanyuan, Qianqian Zhang, Daehan Won, Fake Lu, and Sang Won Yoon. 2020. “Learning Rate Optimization in Convolutional Neural Networks for Medical Images Classification.” In Proceedings of the 2020 IISE Annual Conference

    2020

  22. [30]

    Gaussian Process Regression -Based Learning Rate Optimization in Convolutional Neural Networks for Medical Images Classification

    Li, Yuanyuan, Qianqian Zhang, and Sang Won Yoon. 2021. “Gaussian Process Regression -Based Learning Rate Optimization in Convolutional Neural Networks for Medical Images Classification.” Expert Systems with Applications 184. https://doi.org/10.1016/j.eswa.2021.115357

    2021

  23. [31]

    CAME: Confidence -Guided Adaptive Memory Efficient Optimization

    Luo, Yang, Xiaozhe Ren, Zangwei Zheng, Zhuo Jiang, Xin Jiang, and Yang You. 2023. “CAME: Confidence -Guided Adaptive Memory Efficient Optimization.” In Proceedings of the Annual Meeting of the Association for Computational Linguistics. Vol. 1. https://doi.org/10.18653/v1/2023....

    2023 doi

  24. [32]

    Surrogate Model for Deviation Angle and Total Pressure Loss Prediction of Compressor Based on Machine Learning Methods

    Ma, Bowen, Xiaoxiong Wu, and Yang Yu. 2023. “Surrogate Model for Deviation Angle and Total Pressure Loss Prediction of Compressor Based on Machine Learning Methods.” Hangkong Dongli Xuebao/Journal of Aerospace Power 38 (7). https://doi.org/10.13224/j.cnki.jasp.20220749

    2023 doi

  25. [33]

    A Variational Analysis of Stochastic Gradient Algorithms

    Mandt, Stephan, Matthew D. Hoffman, and David M. Blei. 2016. “A Variational Analysis of Stochastic Gradient Algorithms.” In 33rd International Conference on Machine Learning, ICML 2016. Vol. 1

    2016

  26. [34]

    Stochastic Gradient Descent as Approximate Bayesian Inference

    Mandt, Stephan, Matthew D. Hof Fman, and David M. Blei. 2017. “Stochastic Gradient Descent as Approximate Bayesian Inference.” Journal of Machine Learning Research 18

    2017

  27. [35]

    Artificial Neural Networks for the Prediction of Monkeypox Outbreak

    Manohar, Balakrishnama, and Raja Das. 2022. “Artificial Neural Networks for the Prediction of Monkeypox Outbreak.” Tropical Medicine and Infectious Disease 7 (12). https://doi.org/10.3390/tropicalmed7120424

    2022 doi

  28. [36]

    An Efficient Optimization Technique for Training Deep Neural Networks

    Mehmood, Faisal, Shabir Ahmad, and Taeg Keun Whangbo. 2023. “An Efficient Optimization Technique for Training Deep Neural Networks.” Mathematics 11 (6). https://doi.org/10.3390/math11061360

    2023 doi

  29. [37]

    Grey Wolf Optimizer

    Mirjalili, Seyedali, Seyed Mohammad Mirjalili, and Andrew Lewis. 2014. “Grey Wolf Optimizer.” Advances in Engineering Software 69:46–61. https://doi.org/10.1016/j.advengsoft.2013.12.007

    2014 doi

  30. [38]

    FOX: A FOX -Inspired Optimization Algorithm

    Mohammed, Hardi, and Tarik Rashid. 2023. “FOX: A FOX -Inspired Optimization Algorithm.” Applied Intelligence 53:1030–50. https://doi.org/10.1007/s10489-022-03533-0/Published

    2023 doi

  31. [39]

    A Chaotic Multi -Objective Runge -Kutta Optimization Algorithm for Optimized Circuit Design

    Nyandieka, Owen M., and Davies R. Segera. 2023. “A Chaotic Multi -Objective Runge -Kutta Optimization Algorithm for Optimized Circuit Design.” Mathematical Problems in Engineering 2023. https://doi.org/10.1155/2023/6691214

    2023 doi

  32. [40]

    Ana: Ant Nesting Algorithm for Optimizing Real-World Problems

    Rashid, Deeam Najmadeen Hama, Tarik A. Rashid, and Seyedali Mirjalili. 2021. “Ana: Ant Nesting Algorithm for Optimizing Real-World Problems.” Mathematics 9 (23). https://doi.org/10.3390/math9233111

    2021 doi

  33. [41]

    A Modified Adam Algorithm for Deep Neural Network Optimization

    Reyad, Mohamed, Amany M. Sarhan, and M. Arafa. 2023. “A Modified Adam Algorithm for Deep Neural Network Optimization.” Neural Computing and Applications 35 (23). https://doi.org/10.1007/s00521-023-08568-z

    2023 doi

  34. [42]

    An Overview of Gradient Descent Optimization Algorithms,

    Ruder, Sebastian. 2016. “An Overview of Gradient Descent Optimization Algorithms,” September. http://arxiv.org/abs/1609.04747

    2016 arXiv

  35. [43]

    An Effective Learning Rate Scheduler for Stochastic Gradient Descent- Based Deep Learning Model in Healthcare Diagnosis System

    Sathyabama, K., and K. Saruladha. 2022. “An Effective Learning Rate Scheduler for Stochastic Gradient Descent- Based Deep Learning Model in Healthcare Diagnosis System.” International Journal of Electronic Healthcare 12 (1). https://doi.org/10.1504/ijeh.2022.119587

    2022

  36. [44]

    Training Algorithm Matters for the Performance of Neural Network Potential: Supplementary

    Shao, Yunqi, Florian M Dietrich, Carl Nettelblad, and Chao Zhang. 2021. “Training Algorithm Matters for the Performance of Neural Network Potential: Supplementary.” The Journal of Chemical Physics 155 (20)

    2021

  37. [45]

    Guided Stochastic Gradient Descent Algorithm for Inconsistent Datasets

    Sharma, Anuraganand. 2018. “Guided Stochastic Gradient Descent Algorithm for Inconsistent Datasets.” Applied Soft Computing 73 (December):1068–80. https://doi.org/10.1016/J.ASOC.2018.09.038

    2018 doi

  38. [46]

    Review of Deep Learning Algorithms and Architectures

    Shrestha, Ajay, and Ausif Mahmood. 2019. “Review of Deep Learning Algorithms and Architectures.” IEEE Access. https://doi.org/10.1109/ACCESS.2019.2912200

    2019

  39. [47]

    Don’t Decay the Learning Rate, Increase the Batch Size

    Smith, Samuel L., Pieter Jan Kindermans, Chris Ying, and Quoc V. Le. 2018. “Don’t Decay the Learning Rate, Increase the Batch Size.” In 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings

    2018

  40. [48]

    A Comparison of Optimization Algorithms for Deep Learning

    Soydaner, Derya. 2020. “A Comparison of Optimization Algorithms for Deep Learning.” ArXiv abs/2007.14166. https://api.semanticscholar.org/CorpusID:213116700

    2020 arXiv

  41. [49]

    Automated Requirement Prioritisation Technique Using an Updated Adam Optimisation Algorithm

    Talele, Pratvina, and Rashmi Phalnikar . 2023. “Automated Requirement Prioritisation Technique Using an Updated Adam Optimisation Algorithm.” International Journal of Intelligent Systems and Applications in Engineering 11 (3)

    2023

  42. [50]

    Dynamic Group Optimisation Algorithm for Training Feed -Forward Neural Networks

    Tang, Rui, Simon Fong, Suash Deb, Athanasios V. Vasilakos, and Richard C. Millham. 2018. “Dynamic Group Optimisation Algorithm for Training Feed -Forward Neural Networks.” Neurocomputing 314. https://doi.org/10.1016/j.neucom.2018.03.043. 41

    2018 doi

  43. [51]

    MVI and Forecast Precision Upgrade of Time Series Precipitation Information for Ubiquitous Computing

    Tripathi, Ashok Kumar, P. K. Gupta, Hemraj Saini, and Geetanjali Rathee. 2023. “MVI and Forecast Precision Upgrade of Time Series Precipitation Information for Ubiquitous Computing.” Informatica (Slovenia) 47 (5). https://doi.org/10.31449/INF.V47I5.4152

    2023 doi

  44. [52]

    An Efficient and Effective Second -Order Training Algorithm for LSTM -Based Adaptive Learning

    Vural, N. Mert, Salih Ergut, and Suleyman S. Kozat. 2021. “An Efficient and Effective Second -Order Training Algorithm for LSTM -Based Adaptive Learning.” IEEE Transactions on Signal Processing 69. https://doi.org/10.1109/TSP.2021.3071566

    2021

  45. [53]

    Survey of Digital Image Watermarking Optimization Based on Nature Inspired Algorithms NIAs

    Waleed, Jumana, Huang Dong Jun, Thekra Abbas, Saad Hameed, and Hiyam Hatem. 2014. “Survey of Digital Image Watermarking Optimization Based on Nature Inspired Algorithms NIAs.” International Journal of Security and Its Applications 8 (6). https://doi.org/10.14257/ijsia.2014.8.6.28

    2014 doi

  46. [54]

    Topological Measurement of Deep Neural Networks Using Persistent Homology

    Watanabe, Satoru, and Hayato Yamana. 2022. “Topological Measurement of Deep Neural Networks Using Persistent Homology.” Annals of Mathematics and Artificial Intelligence 90 (1). https://doi.org/10.1007/s10472- 021-09761-3

    2022 doi

  47. [55]

    DPIS: An Enhanced Mechanism for Differentially Private SGD with Importance Sampling

    Wei, Jianxin, Ergute Bao, Xiaokui Xiao, and Yin Yang. 2022. “DPIS: An Enhanced Mechanism for Differentially Private SGD with Importance Sampling.” In Proceedings of the ACM Conference on Computer and Communications Security. https://doi.org/10.1145/3548606.3560562

    2022

  48. [56]

    DIRECTION MATTERS: ON THE IMPLICIT BIAS OF STOCHASTIC GRADIENT DESCENT WITH MODERATE LEARNING RATE

    Wu, Jingfeng, Vladimir Braverman, Difan Zou, and Quanquan Gu. 2021. “DIRECTION MATTERS: ON THE IMPLICIT BIAS OF STOCHASTIC GRADIENT DESCENT WITH MODERATE LEARNING RATE.” In ICLR 2021 - 9th International Conference on Learning Representations

    2021

  49. [57]

    On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice

    Yang, Li, and Abdallah Shami. 2020. “On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice.” Neurocomputing 415. https://doi.org/10.1016/j.neucom.2020.07.061

    2020 doi

  50. [58]

    An Improved CNN Model for Fast Salient Object Detection

    Zhang, Bin, Yang Wu, Jiaqiang Zhang, and Ming Ma. 2021. “An Improved CNN Model for Fast Salient Object Detection.” In Communications in Computer and Information Science. Vol. 1423. https://doi.org/10.1007/978-3- 030-78618-2_6

    2021 doi

  51. [59]

    Hartley Spectral Pooling for Deep Learning

    Zhang, Hao, and Jianwei Ma. 2020. “Hartley Spectral Pooling for Deep Learning.” CSIAM Transactions on Applied Mathematics 1 (3). https://doi.org/10.4208/csiam-am.2020-0018

    2020 doi

  52. [60]

    Deconstructing What Makes a Good Optimizer for Language Models

    Zhao, Rosie, Depen Morwani, David Brandfonbrener, Nikhil Vyas, and Sham M Kakade. 2024. “Deconstructing What Makes a Good Optimizer for Language Models.” ArXiv abs/2407.07972. https://api.semanticscholar.org/CorpusID:271097803

    2024 arXiv

  53. [61]

    Carbon Price Forecasting Based on CEEMDAN and LSTM

    Zhou, Feite, Zhehao Huang, and Changhong Zhang. 2022. “Carbon Price Forecasting Based on CEEMDAN and LSTM.” Applied Energy 311. https://doi.org/10.1016/j.apenergy.2022.118601

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.