Pith. sign in

REVIEW 4 major objections 5 minor 53 references

Adaptive Learning of Design Strategies over Non-Hierarchical Multi-Fidelity Models via Policy Alignment

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read No model ladder: adaptive RL borrows cheap simulators in place

desk verdict ALPHA is a genuinely new adaptive multi-fidelity RL scheme, clearly described and worth a serious referee, but its core alignment-to-reward assumption is unvalidated and the empirical evidence lacks repeated runs and statistics. read the letter →

arxiv 2411.10841 v1 pith:EI45OZ7F submitted 2024-11-16 cs.LG cs.AI

Akash Agrawal (1) , Christopher McComb (1) ((1) Carnegie Mellon University) This is my paper · ORCID
classification cs.LGcs.AI
keywords multi-fidelityreinforcementlearningheterogeneousmodelspolicyalignmentadaptivemethodsnon-hierarchicalframeworksengineeringdesignoctocopterproximaloptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes ALPHA, a multi-fidelity reinforcement learning framework that learns a high-fidelity design policy by adaptively borrowing from a set of non-hierarchical, heterogeneous low-fidelity models. Instead of following a fixed low-to-high schedule, ALPHA watches how closely each low-fidelity policy's action choices match the high-fidelity policy's choices, and uses the cheap models mainly in the design regions where they agree. The claim is that the resulting unified high-fidelity policy matches or beats hierarchical multi-fidelity RL baselines at lower computational expense, and that adaptive agents take more direct paths to high-performance designs. The paper demonstrates this in an Ackley-style analytical optimization problem and an octocopter design task with two low-fidelity surrogates and one high-fidelity flight simulator.

What carries the argument

The load-bearing mechanism is Algorithm 1, ALPHA, built around a policy-alignment gate. For each low-fidelity agent, cosine similarity between the mean of its action distribution and the mean of the high-fidelity policy's action distribution at the current state is computed and compared with a threshold theta that follows a cosine schedule from 90 to 0 degrees over training. Algorithm 2 converts the two similarity comparisons into an epsilon-greedy categorical choice over LF1, LF2, and HF; after each episode, contiguous run-lengths of low-fidelity experience in which the chosen model was aligned and exploitatively selected are appended to the HF buffer, so the HF policy trains on LF rewards only in the regions the alignment gate certifies.

What would settle it

In a two-dimensional test problem, compute the true per-region difference between each low-fidelity model's reward and the high-fidelity model's reward, then compare it against the cosine alignment used by ALPHA; if regions with high alignment show no smaller reward error than unaligned regions, the central efficiency claim collapses. A simpler version: build a low-fidelity model whose action guidance agrees with the high-fidelity policy but whose reward values systematically overestimate performance, and observe whether ALPHA's final policy degrades.

Watch

Extended reading notes

Core claim

The central discovery is that the alignment between low-fidelity and high-fidelity policies, measured by the cosine similarity of the means of their action distributions at a given state, is a sufficient guide for deciding when a low-fidelity model's rewards can safely contribute to training the high-fidelity policy. ALPHA maintains one policy per analysis model, picks a model at each design step with an epsilon-greedy rule whose probabilities depend on this alignment against a cosine-annealed threshold, and augments the high-fidelity replay buffer with contiguous subsequences of low-fidelity experience collected in aligned regions. On the two case studies, the trained high-fidelity policy attains consistently high solution quality and lower evaluation cost than training on the high-fidelity model alone, while avoiding the ordering sensitivity that afflicts hierarchical schedules. The paper concludes that adaptive policy alignment eliminates the need for a predefined model schedule and yields more direct convergence paths.

Load-bearing premise

The method assumes that when a low-fidelity policy's action distribution points in the same direction as the high-fidelity policy's, the low-fidelity model's rewards are trustworthy enough to train the high-fidelity policy; if action agreement does not imply reward accuracy, the high-fidelity policy can be trained on biased low-fidelity rewards.

Editorial extensions

If this is right

  • A designer can combine arbitrary low-fidelity models with a high-fidelity one and obtain a single high-fidelity policy without deciding a model schedule in advance.
  • The learned policy's convergence does not depend on the order in which the low-fidelity models are presented, unlike hierarchical multi-fidelity RL baselines.
  • Low-fidelity models are used for broad exploration while the high-fidelity model is reserved for critical regions such as the global optimum, concentrating computation where accuracy matters.
  • If the alignment signal is reliable, the same framework should extend to any number of non-hierarchical models with heterogeneous error distributions.
  • Solution quality is maintained while evaluation time is reduced relative to high-fidelity-only training, and variance across seeds is lower than in the baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test the paper does not run: compare the alignment score against the actual per-region reward error of each low-fidelity model; if trajectories with high cosine similarity do not have lower reward error, the data-augmentation step is training on biased rewards.
  • Because model choice probabilities are not cost-weighted, replacing them with cost-aware weights when several low-fidelity models align could further improve the quality-efficiency tradeoff.
  • The alignment threshold schedule is hand-tuned via a cosine rule; an adaptive threshold driven by measured regional agreement could make the method robust to problems where alignment evolves non-monotonically.
  • The framework assumes every model shares a common design representation; extending it to models with different state and action spaces would require a representation alignment step beyond policy alignment.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes ALPHA, a multi-fidelity reinforcement learning framework for engineering design that adaptively combines one high-fidelity (HF) model with several non-hierarchical low-fidelity (LF) models. Each model has its own policy; at each design step, the framework computes the cosine similarity between the mean actions of each LF policy and the HF policy, uses an epsilon-greedy rule with a cosine-scheduled alignment threshold to select which model to step, and then augments the HF replay buffer with aligned contiguous subsequences of LF experience. The method is evaluated on a modified Ackley optimization problem and an octocopter design problem, with two LF models in each case. The central claims are that ALPHA eliminates the need for a predefined model schedule, learns a unified HF policy that yields high-quality solutions at reduced computational expense, and shows superior convergence behavior compared to hierarchical multi-fidelity RL baselines.

Significance. If the claims are substantiated, ALPHA would be a useful contribution to multi-fidelity RL in engineering design: it relaxes the hierarchy assumption that underlies most existing multi-fidelity RL methods, and it introduces a concrete mechanism for leveraging heterogeneous LF models via policy alignment. The paper is clearly written, the algorithm is explicit, and the two case studies provide qualitative evidence of adaptive model usage across time and design space, including Moran's I analysis. However, the central efficiency and quality claims currently rest on an unvalidated proxy (cosine similarity of policy action means implying reward accuracy in aligned regions) and on results from what appear to be single training runs without statistical replication. These issues, along with the low global accuracy of the octocopter surrogates, make the empirical support conditional rather than conclusive.

major comments (4)
  1. [§3.1, Algorithm 1 and data augmentation paragraph] The load-bearing assumption is that cosine similarity between the means of the action distributions of an LF policy and the HF policy is a reliable signal that LF rewards can be used to train the HF policy. The text states that 'rewards from LF models are used in aligned regions of a trajectory' and that this 'maintains the learning precision of the HF policy,' but cosine similarity measures policy agreement, not reward or value accuracy. If LF rewards are biased in an aligned region, the HF policy is trained on biased targets while still being called a high-fidelity policy. The paper provides no direct test of this correspondence. Please add a validation: for example, measure LF reward error (or value error) in aligned versus non-aligned regions, and/or compare ALPHA against an ablation that does not augment LF experience into the HF buffer. Without such evidence, the efficiency and quality results are only conditionally supported.
  2. [§5.1 and §5.2, Figures 9 and 15] The results appear to come from a single training run per condition; the violin plots show variability over seed designs, not over stochastic training runs with different random seeds. The conclusion claims that ALPHA 'consistently yields high quality solutions,' but with one run per condition there is no statistical basis for consistency. Please report multiple independent training runs (e.g., 5–10 random seeds) and apply appropriate statistical tests on final HF-evaluated solution quality, such as Mann–Whitney U tests or bootstrap confidence intervals. Also report the random seeds and the full set of PPO and environment hyperparameters to enable reproducibility.
  3. [§4.2, octocopter design problem] The low-fidelity surrogates LF1 and LF2 have global prediction accuracies of 0.23 and 0.38 on the entire dataset (presumably R-squared, but the metric is not defined). These values indicate that the surrogates are very inaccurate globally, which makes it especially important to demonstrate that the LF rewards used in aligned regions are locally accurate. The paper only reports validation accuracies on the respective training subsets (0.51 and 0.56) and does not report local accuracy in the regions where each LF model is actually selected or aligned. Please report local reward error or R-squared in the aligned regions, or otherwise show that the injected LF rewards are not systematically biased in those regions.
  4. [§3.1, Algorithm 2 and Figures 11, 18] The cosine schedule for the alignment threshold and the epsilon-greedy parameter are free hyperparameters that directly control the four regimes R1–R4 observed in the model-usage plots. Without a sensitivity analysis or ablation (e.g., fixed threshold, different schedule slopes, different epsilon values), it is unclear how much of the claimed adaptivity is due to the method itself versus the hand-chosen schedule. Please add an ablation or sensitivity study over a small grid of these parameters, at least for the analytical problem, to support the generality of the adaptive behavior.
minor comments (5)
  1. [§4] The subsection numbering is inconsistent: the octocopter problem is labeled '4.1' but should be '4.2' after the analytical problem.
  2. [§5.2, Figures 15–17] The text refers inconsistently to Figures 15 and 16: for example, the caption is 'FIGURE 16' but the text calls it 'Fig. 16(a)' in some places and 'Fig. 15(c)' in others. Please unify the figure numbering and cross-references.
  3. [Algorithm 2] The definition of 'aligned' is terse: 'aligned = [model == (argmax p_m)]'. This means that if the epsilon-greedy exploration selects a non-argmax model, aligned is False, but the data augmentation paragraph only says 'chosen exploitatively.' Please state this explicitly in the main text to avoid ambiguity.
  4. [§5.2, computational cost] The HF simulator cost in the octocopter problem has a heavy-tailed distribution (25th percentile 0 s, median 1.78 s, 75th percentile 22.04 s). This variability is likely to affect the efficiency comparison; consider reporting total compute with confidence intervals or using a cost metric that is robust to outliers.
  5. [References] Reference [21] lacks year and venue information.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: ALPHA's central claims are evaluated against held-out seeds and pre-constructed models; the only self-citation is a non-load-bearing reuse of the authors' prior hierarchical framework.

full rationale

The paper's derivation chain is self-contained for its central comparisons. The low-fidelity models are fixed before training (Ackley LF1/LF2 are closed-form functions with shifted minima; octocopter LF1/LF2 are neural-network surrogates trained on distinct partial-convergence datasets), and no parameter of ALPHA is fitted to the final high-fidelity solution quality. The HF policy is trained with PPO using HF and aligned-LF data, then evaluated on held-out seed designs with the HF model, so the quality claim is not defined into the algorithm. The reduced computational expense follows from the measured per-model evaluation costs plus the recorded model-usage proportions, not from an identity with the output. The paper does cite the authors' prior hierarchical framework [3] for the reward structure and as a baseline, but the adaptive alignment mechanism, threshold schedule, and model-choice rules are new and are not justified by that citation; the prior work is used as a comparison method. The main weakness is the unvalidated proxy that cosine similarity between action-distribution means marks regions where LF rewards can safely augment HF training; the paper admits alignment metrics remain open in future work. That is an internal-validity/correctness concern, not a circular reduction: the paper does not define 'aligned' to mean 'reward-accurate,' and it does not use this proxy as its outcome metric. No equation or construction makes a predicted quantity equal to an input by definition.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The framework introduces no new physical entities. Its central claim rests on hand-chosen procedural parameters and two untested proxy assumptions: that policy alignment implies reward fidelity and that aligned low-fidelity experience can safely augment high-fidelity training.

free parameters (5)
  • Alignment threshold cosine schedule = 90 to 0 degrees over 90% of training
    Hand-chosen schedule in Algorithm 2; controls how quickly low-fidelity models are abandoned. No sensitivity analysis is reported.
  • Epsilon-greedy exploration constant = Not reported
    Used in Algorithm 2 to balance exploration among models; value is not specified.
  • PPO hyperparameters = Not reported
    Learning rate, discount factor, GAE lambda, clip range, epochs, and batch size are not given, making exact reimplementation impossible.
  • Episode length and episode count = 20 steps; 300 and 1200 episodes
    Chosen per case study without justification; these budgets affect the convergence comparison between agents.
  • Policy and value network architectures = I2-D1024-R-D1024-R outputs (Ackley); I4-D1024-R-D1024-R outputs (octocopter)
    Network sizes and activations are specified but not ablated; all agents use the same architecture.
assumptions (4)
  • domain assumption Each simulator operates on a common design representation.
    Stated in the conclusion as an assumption; ALPHA requires one shared state and action space across all agents.
  • domain assumption Cosine similarity between mean actions of low-fidelity and high-fidelity policies is a valid measure of alignment.
    Used in Algorithm 1 and Algorithm 2 to drive model selection, with no independent validation that this proxy tracks model utility.
  • domain assumption Low-fidelity rewards in aligned regions can be used to train the high-fidelity policy without introducing harmful bias.
    Invoked in the data augmentation paragraph of Section 3.1; if false, the high-fidelity policy would be trained on inaccurate rewards.
  • domain assumption The high-fidelity model is the ground truth for evaluating solution quality.
    All final quality metrics and trajectories are evaluated with the high-fidelity simulator, which is standard but assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adaptive Learning of Design Strategies over Non-Hierarchical Multi-Fidelity Models via Policy Alignment." pith.science (2026). https://pith.science/paper/EI45OZ7F

@misc{pith2026241110841,
  author       = {Pith},
  title        = {Pith review of: Adaptive Learning of Design Strategies over Non-Hierarchical Multi-Fidelity Models via Policy Alignment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EI45OZ7F}},
  note         = {Machine review of arXiv:2411.10841}
}
read the original abstract

Multi-fidelity Reinforcement Learning (RL) frameworks significantly enhance the efficiency of engineering design by leveraging analysis models with varying levels of accuracy and computational costs. The prevailing methodologies, characterized by transfer learning, human-inspired strategies, control variate techniques, and adaptive sampling, predominantly depend on a structured hierarchy of models. However, this reliance on a model hierarchy overlooks the heterogeneous error distributions of models across the design space, extending beyond mere fidelity levels. This work proposes ALPHA (Adaptively Learned Policy with Heterogeneous Analyses), a novel multi-fidelity RL framework to efficiently learn a high-fidelity policy by adaptively leveraging an arbitrary set of non-hierarchical, heterogeneous, low-fidelity models alongside a high-fidelity model. Specifically, low-fidelity policies and their experience data are dynamically used for efficient targeted learning, guided by their alignment with the high-fidelity policy. The effectiveness of ALPHA is demonstrated in analytical test optimization and octocopter design problems, utilizing two low-fidelity models alongside a high-fidelity one. The results highlight ALPHA's adaptive capability to dynamically utilize models across time and design space, eliminating the need for scheduling models as required in a hierarchical framework. Furthermore, the adaptive agents find more direct paths to high-performance solutions, showing superior convergence behavior compared to hierarchical agents.

Figures

Figures reproduced from arXiv: 2411.10841 by the authors.

Figure 1
Figure 1. FIGURE 1: A [PITH_FULL_IMAGE:figures/full_fig_p011_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 29 canonical work pages

  1. [1]

    Multi-Fidelity Reinforcement Learning Framework for Shape Optimization,

    Bhola, S., Pawar, S., Balaprakash, P., and Maulik, R., 2023, “Multi-Fidelity Reinforcement Learning Framework for Shape Optimization,” J Comput Phys,

    work page 2023

  2. [2]

    Multi-Fidelity Optimization of a Quiet Propeller Based on Deep Deterministic Policy Gradient and Transfer Learning,

    Geng, X., Liu, P., Hu, T., Qu, Q., Dai, J., Lyu, C., Ge, Y., and Akkermans, R. A. D., 2023, “Multi-Fidelity Optimization of a Quiet Propeller Based on Deep Deterministic Policy Gradient and Transfer Learning,” Aerosp Sci Technol, 137. https://doi.org/10.1016/j.ast.2023.108288

    arXiv 2023

  3. [3]

    Reinforcement Learning for Efficient Design Space Exploration With Variable Fidelity Analysis Models,

    Agrawal, A., and McComb, C., 2023, “Reinforcement Learning for Efficient Design Space Exploration With Variable Fidelity Analysis Models,” J Comput Inf Sci Eng, 23(4). https://doi.org/10.1115/1.4056297

    work page doi:10.1115/1.4056297 2023

  4. [4]

    Multifidelity Reinforcement Learning with Control Variates,

    Khairy, S., and Balaprakash, P., 2022, “Multifidelity Reinforcement Learning with Control Variates,” Neurocomputing, 597, p. 127963. https://doi.org/10.1016/j.neucom.2024.127963

    work page arXiv 2022

  5. [5]

    Leveraging Deep Reinforcement Learning for Design Space Exploration with Multi-Fidelity Surrogate Model,

    Li, H., Wang, R., Wang, Z., Li, G., Wang, G., and Yan, Y., 2024, “Leveraging Deep Reinforcement Learning for Design Space Exploration with Multi-Fidelity Surrogate Model,” Journal of Engineering Design, pp. 1–40. https://doi.org/10.1080/09544828.2024.2366686

    arXiv 2024

  6. [6]

    On the Use of Surrogate Models in Engineering Design Optimization and Exploration: The Key Issues,

    Palar, P. S., Zuhal, L. R., Liem, R. P., and Shimoyama, K., 2019, “On the Use of Surrogate Models in Engineering Design Optimization and Exploration: The Key Issues,” GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion, Association for Computing Machinery, Inc, New York, NY, USA, pp. 1592–1602. https://...

    arXiv 2019

  7. [7]

    A Review of Surrogate Modeling Techniques for Aerodynamic Analysis and Optimization: Current Limitations and Future Challenges in Industry,

    Yondo, R., Bobrowski, K., Andrés, E., and Valero, E., 2019, “A Review of Surrogate Modeling Techniques for Aerodynamic Analysis and Optimization: Current Limitations and Future Challenges in Industry,” Advances in Evolutionary and Deterministic Methods for Design, Optimization and Control in Engineering and Sciences, E. Minisci, M. Vasile, J. Periaux, N.R...

    work page doi:10.1007/978-3-319-8 2019

  8. [8]

    Review of Multi-Fidelity Models,

    Fernández-Godino, M. G., Park, C., Kim, N.-H., and Haftka, R. T., 2016, “Review of Multi-Fidelity Models,” arXiv preprint arXiv:1609.07196, 1(4), pp. 351–400. https://doi.org/10.2514/1.J057750

    arXiv 2016

Show all 53 references
  1. [9]

    Survey of Multifidelity Methods in Uncertainty Propagation, Inference, and Optimization,

    Peherstorfer, B., Willcox, K., and Gunzburger, M., 2018, “Survey of Multifidelity Methods in Uncertainty Propagation, Inference, and Optimization,” SIAM Review, 60(3), pp. 550–591. https://doi.org/10.1137/16M1082469

    2018 doi

  2. [10]

    Surrogate Modeling: Tricks That Endured the Test of Time and Some Recent Developments,

    Viana, F. A. C., Gogu, C., and Goel, T., 2021, “Surrogate Modeling: Tricks That Endured the Test of Time and Some Recent Developments,” Structural and Multidisciplinary Optimization, 64(5), pp. 2881–2908. https://doi.org/10.1007/s00158-021-03001-2

    2021 doi

  3. [11]

    A Multi-Fidelity Surrogate Modeling Method in the Presence of Non-Hierarchical Low-Fidelity Data,

    Chen, H., Ouyang, L., Liu, L., and Ma, Y., 2024, “A Multi-Fidelity Surrogate Modeling Method in the Presence of Non-Hierarchical Low-Fidelity Data,” Aerosp Sci Technol, 146. https://doi.org/10.1016/j.ast.2024.108928. 45

    2024

  4. [12]

    Extended Hierarchical Kriging Method for Aerodynamic Model Generation Incorporating Multiple Low- Fidelity Datasets,

    Pham, V., Tyan, M., Nguyen, T. A., and Lee, J. W., 2024, “Extended Hierarchical Kriging Method for Aerodynamic Model Generation Incorporating Multiple Low- Fidelity Datasets,” Aerospace, 11(1). https://doi.org/10.3390/aerospace11010006

    2024 doi

  5. [13]

    A Latent Variable Approach for Non-Hierarchical Multi-Fidelity Adaptive Sampling,

    Chen, Y.-P., Wang, L., Comlek, Y., and Chen, W., 2024, “A Latent Variable Approach for Non-Hierarchical Multi-Fidelity Adaptive Sampling,” Comput Methods Appl Mech Eng, 421, p. 116773. https://doi.org/10.1016/j.cma.2024.116773

    2024

  6. [14]

    Deriving Metamodels to Relate Machine Learning Quality to Design Repository Characteristics in the Context of Additive Manufacturing,

    Williams, G., Meisel, N. A., Simpson, T. W., and McComb, C., 2020, “Deriving Metamodels to Relate Machine Learning Quality to Design Repository Characteristics in the Context of Additive Manufacturing,” Volume 11A: 46th Design Automation Conference (DAC), American Society of M...

    2020 doi

  7. [15]

    Design Repository Effectiveness for 3D Convolutional Neural Networks: Application to Additive Manufacturing,

    Williams, G., Meisel, N. A., Simpson, T. W., and McComb, C., 2019, “Design Repository Effectiveness for 3D Convolutional Neural Networks: Application to Additive Manufacturing,” Journal of Mechanical Design, 141(11). https://doi.org/10.1115/1.4044199

    2019 doi

  8. [16]

    Comparing Attribute- and Form-Based Machine Learning Techniques for Component Prediction,

    Williams, G., Puentes, L., Nelson, J., Menold, J., Tucker, C., and McComb, C., 2020, “Comparing Attribute- and Form-Based Machine Learning Techniques for Component Prediction,” Volume 11B: 46th Design Automation Conference (DAC), American Society of Mechanical Engineers. https...

    2020 doi

  9. [17]

    Fairness- and Uncertainty- Aware Data Generation for Data-Driven Design,

    Xie, J., Zhang, C., Sun, L., and Zhao, Y. F., 2023, “Fairness- and Uncertainty- Aware Data Generation for Data-Driven Design,” Volume 2: 43rd Computers and Information in Engineering Conference (CIE), American Society of Mechanical Engineers. https://doi.org/10.1115/DETC2023-114687

    2023 doi

  10. [18]

    Recent Advances in Surrogate Modeling Methods for Uncertainty Quantification and Propagation,

    Wang, C., Qiang, X., Xu, M., and Wu, T., 2022, “Recent Advances in Surrogate Modeling Methods for Uncertainty Quantification and Propagation,” Symmetry (Basel), 14(6). https://doi.org/10.3390/sym14061219

    2022 doi

  11. [19]

    Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations,

    Raissi, M., Perdikaris, P., and Karniadakis, G. E., 2019, “Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations,” J Comput Phys, 378, pp. 686–707. https://doi.org/10.1016/j.jcp.2...

    2019 doi

  12. [20]

    https://doi.org/10.1007/978-981-15-4095-0

    Dong, H., Ding, Z., and Zhang, S., 2020, Deep Reinforcement Learning, Springer Singapore, Singapore. https://doi.org/10.1007/978-981-15-4095-0

    2020 doi

  13. [21]

    S., and Barto, A

    Sutton, R. S., and Barto, A. G., Reinforcement Learning : An Introduction

  14. [22]

    Human-Level Control through Deep Reinforcement Learning,

    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., and Hassabis, D., 2015, “Human-Le...

    2015 doi

  15. [23]

    Mastering the Game of Go with Deep Neural Networks and Tree Search,

    Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., a...

    2016 doi

  16. [24]

    Discrete Structural Design Synthesis: A Hierarchical-Inspired Deep Reinforcement Learning Approach Considering Topological and Parametric Actions,

    Ororbia, M. E., and Warn, G. P., 2024, “Discrete Structural Design Synthesis: A Hierarchical-Inspired Deep Reinforcement Learning Approach Considering Topological and Parametric Actions,” Journal of Mechanical Design, pp. 1–16. https://doi.org/10.1115/1.4065488

    2024 doi

  17. [25]

    Reimagining Space Layout Design through Deep Reinforcement Learning,

    Kakooee, R., and Dillenburger, B., 2024, “Reimagining Space Layout Design through Deep Reinforcement Learning,” J Comput Des Eng, 11(3), pp. 43–55. https://doi.org/10.1093/jcde/qwae025

    2024 doi

  18. [26]

    Reinforcement Learning for Engineering Design Automation,

    Dworschak, F., Dietze, S., Wittmann, M., Schleich, B., and Wartzack, S., 2022, “Reinforcement Learning for Engineering Design Automation,” Advanced Engineering Informatics, 52. https://doi.org/10.1016/j.aei.2022.101612

    2022

  19. [27]

    https://doi.org/10.1016/j.matdes.2022.110672

    Brown, N., Garland, A., Fadel, G., and Li, G., 2022, Deep Reinforcement Learning for Engineering Design through Topology Optimization of Elementally Discretized Design Domains. https://doi.org/10.1016/j.matdes.2022.110672

    2022

  20. [28]

    A Deep Reinforcement Learning Approach for Global Routing,

    Liao, H., Zhang, W., Dong, X., Poczos, B., Shimada, K., and Burak Kara, L., 2020, “A Deep Reinforcement Learning Approach for Global Routing,” Journal of Mechanical Design, 142(6). https://doi.org/10.1115/1.4045044

    2020 doi

  21. [29]

    GCP-HOLO: Generating High- Order Linkage Graphs for Path Synthesis,

    Fogelson, M. B., Tucker, C., and Cagan, J., 2023, “GCP-HOLO: Generating High- Order Linkage Graphs for Path Synthesis,” Journal of Mechanical Design, 145(7). https://doi.org/10.1115/1.4062147

    2023 doi

  22. [30]

    Generative Design by Reinforcement Learning: Enhancing the Diversity of Topology Optimization Designs,

    Jang, S., Yoo, S., and Kang, N., 2022, “Generative Design by Reinforcement Learning: Enhancing the Diversity of Topology Optimization Designs,” CAD Computer Aided Design, 146, p. 103225. https://doi.org/10.1016/j.cad.2022.103225

    2022

  23. [31]

    Modular Robot Design Synthesis with Deep Reinforcement Learning,

    Whitman, J., Bhirangi, R., Travers, M., and Choset, H., 2020, “Modular Robot Design Synthesis with Deep Reinforcement Learning,” Proceedings of the AAAI Conference on Artificial Intelligence, 34(06), pp. 10418–10425. https://doi.org/10.1609/aaai.v34i06.6611

    2020 doi

  24. [32]

    Learning to Design Without Prior Data: Discovering Generalizable Design Strategies Using Deep Learning and Tree Search,

    Raina, A., Cagan, J., and McComb, C., 2023, “Learning to Design Without Prior Data: Discovering Generalizable Design Strategies Using Deep Learning and Tree Search,” Journal of Mechanical Design, 145(3). https://doi.org/10.1115/1.4056221

    2023 doi

  25. [33]

    A Cost-Aware Multi-Agent System for Black-Box Design Space Exploration,

    Chen, S., Bayrak, A. E., and Sha, Z., 2025, “A Cost-Aware Multi-Agent System for Black-Box Design Space Exploration,” Journal of Mechanical Design, 147(1), pp. 1–17. https://doi.org/10.1115/1.4065914

    2025 doi

  26. [34]

    A Case Study of Deep Reinforcement Learning for Engineering Design: Application to Microfluidic Devices for Flow Sculpting,

    Lee, X. Y., Balu, A., Stoecklein, D., Ganapathysubramanian, B., and Sarkar, S., 2019, “A Case Study of Deep Reinforcement Learning for Engineering Design: Application to Microfluidic Devices for Flow Sculpting,” Journal of Mechanical Design, Transactions of the ASME, 141(11). ...

    2019 doi

  27. [35]

    Framework for Design Optimization Using Deep Reinforcement Learning,

    Yonekura, K., and Hattori, H., 2019, “Framework for Design Optimization Using Deep Reinforcement Learning,” Structural and Multidisciplinary Optimization, 60(4), pp. 1709–1713. https://doi.org/10.1007/s00158-019-02276-w

    2019 doi

  28. [36]

    Analyzing Real Options and Flexibility in Engineering Systems Design Using Decision Rules and Deep Reinforcement Learning,

    Caputo, C., and Cardin, M. A., 2022, “Analyzing Real Options and Flexibility in Engineering Systems Design Using Decision Rules and Deep Reinforcement Learning,” Journal of Mechanical Design, 144(2). https://doi.org/10.1115/1.4052299. 47

    2022 doi

  29. [37]

    The Computational Limits of Deep Learning,

    Thompson, N., Greenewald, K., Lee, K., and Manso, G. F., 2020, “The Computational Limits of Deep Learning,” Ninth Computing within Limits 2023, LIMITS. https://doi.org/10.21428/bf6fb269.1f033948

    2020 doi

  30. [38]

    Between Progress and Potential Impact of AI: The Neglected Dimensions,

    Martínez-Plumed, F., Avin, S., Brundage, M., Dafoe, A., hÉigeartaigh, S. Ó., and Hernández-Orallo, J., 2018, “Between Progress and Potential Impact of AI: The Neglected Dimensions,” arXiv preprint arXiv:1806.00610. [Online]. Available: http://arxiv.org/abs/1806.00610

    2018 arXiv

  31. [39]

    On the Effects of Heterogeneous Errors on Multi-Fidelity Bayesian Optimization

    Foumani, Z. Z., Yousefpour, A., Shishehbor, M., and Bostanabad, R., 2023, “On the Effects of Heterogeneous Errors on Multi-Fidelity Bayesian Optimization.” https://doi.org/10.48550/arXiv.2309.02771

    2023 doi arXiv

  32. [40]

    Nonhierarchical Multi‐model Fusion Using Spatial Random Processes,

    Chen, S., Jiang, Z., Yang, S., Apley, D. W., and Chen, W., 2016, “Nonhierarchical Multi‐model Fusion Using Spatial Random Processes,” Int J Numer Methods Eng, 106(7), pp. 503–526. https://doi.org/10.1002/nme.5123

    2016 doi

  33. [41]

    Predicting the Output from a Complex Computer Code When Fast Approximations Are Available,

    Kennedy, M., 2000, “Predicting the Output from a Complex Computer Code When Fast Approximations Are Available,” Biometrika, 87(1), pp. 1–13. https://doi.org/10.1093/biomet/87.1.1

    2000 doi

  34. [42]

    Reinforcement Learning with Multi-Fidelity Simulators,

    Cutler, M., Walsh, T. J., and How, J. P., 2014, “Reinforcement Learning with Multi-Fidelity Simulators,” 2014 IEEE International Conference on Robotics and Automation (ICRA), IEEE, pp. 3888–3895. https://doi.org/10.1109/ICRA.2014.6907423

    2014

  35. [43]

    Multifidelity Reinforcement Learning with Gaussian Processes: Model-Based and Model-Free Algorithms,

    Suryan, V., Gondhalekar, N., and Tokekar, P., 2020, “Multifidelity Reinforcement Learning with Gaussian Processes: Model-Based and Model-Free Algorithms,” IEEE Robot Autom Mag, 27(2), pp. 117–128. https://doi.org/10.1109/MRA.2020.2977971

    2020

  36. [44]

    Toward Multi-Fidelity Reinforcement Learning for Symbolic Optimization,

    Silva, F. L., Yang, J., Landajuela, M., Goncalves, A., Ladd, A., Faissol, D., and Petersen, B., 2023, “Toward Multi-Fidelity Reinforcement Learning for Symbolic Optimization,” Workshop on Adaptive and Learning Agents Toward Multi-Fidelity Reinforcement Learning for Symbolic Op...

    2023

  37. [45]

    Low- Cost Multi-Agent Navigation via Reinforcement Learning with Multi-Fidelity Simulator,

    Qiu, J., Yu, C., Liu, W., Yang, T., Yu, J., Wang, Y., and Yang, H., 2021, “Low- Cost Multi-Agent Navigation via Reinforcement Learning with Multi-Fidelity Simulator,” IEEE Access, 9, pp. 84773–84782. https://doi.org/10.1109/ACCESS.2021.3085328

    2021

  38. [46]

    SGDR: Stochastic Gradient Descent with Warm Restarts

    Loshchilov, I., and Hutter, F., 2016, “SGDR: Stochastic Gradient Descent with Warm Restarts.” [Online]. Available: http://arxiv.org/abs/1608.03983

    2016 arXiv

  39. [47]

    Proximal Policy Optimization Algorithms,

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O., 2017, “Proximal Policy Optimization Algorithms,” arXiv preprint arXiv:1707.06347. [Online]. Available: http://arxiv.org/abs/1707.06347

    2017 arXiv

  40. [49]

    Moran, P. A. P., 1950, Biometrika Trust Notes on Continuous Stochastic Phenomena. [Online]. Available: https://www.jstor.org/stable/2332142

    1950

  41. [50]

    Ackley, D., 2012, A Connectionist Machine for Genetic Hillclimbing, Springer science & business media. 48

    2012

  42. [51]

    A High-Fidelity Simulation Test-Bed for Fault-Tolerant Octo-Rotor Control Using Reinforcement Learning,

    Ahmed, I., Quinones-Grueiro, M., and Biswas, G., 2022, “A High-Fidelity Simulation Test-Bed for Fault-Tolerant Octo-Rotor Control Using Reinforcement Learning,” AIAA/IEEE Digital Avionics Systems Conference - Proceedings, Institute of Electrical and Electronics Engineers Inc.,...

    2022

  43. [52]

    A Flight Dynamics Model for Exploring the Distributed Electrical EVTOL Cyber Physical Design Space,

    Walker, J. D., Heim, F. M., Surampudi, B., Bueno, P., Carpenter, A., Chocron, S., Cutshall, J., Lammons, R., Bapty, T., Swenson, B., and Whittington, S., 2022, “A Flight Dynamics Model for Exploring the Distributed Electrical EVTOL Cyber Physical Design Space,” Proceedings - 4...

    2022

  44. [53]

    Martins, J. R. R. A., and Ning, A., 2021, Engineering Design Optimization, Cambridge University Press. https://doi.org/10.1017/9781108980647

    2021 doi

  45. [482]

    https://doi.org/10.1016/j.jcp.2023.112018

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.