Pith. sign in

REVIEW 4 major objections 4 minor 32 references

Multiple-Frequencies Population-Based Training

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PBT's long-run collapse comes from a single evolution frequency, and MF-PBT fixes it with fast and slow sub-populations joined by asymmetric migration.

desk verdict A solid incremental PBT variant with a real new idea (multi-frequency sub-populations plus asymmetric migration), but the 'always' claim and the n=8 steady-population mechanism need tightening before it becomes a reliable recommendation. read the letter →

arxiv 2506.03225 v2 pith:P4U7X2YU submitted 2025-06-03 cs.LG cs.AIcs.NE

Waël Doulazmi , Auguste Lehuger , Marin Toromanoff , Valentin Charraut , Thibault Buhet , Fabien Moutarde This is my paper
classification cs.LGcs.AIcs.NE
keywords population-basedtraininghyperparameteroptimizationreinforcementlearningevolutionfrequencygreedinessmulti-frequencyBraxvarianceexploitation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Population-Based Training (PBT) is a widely used way to tune hyperparameters while training reinforcement-learning agents, but its habit of selecting on recent performance makes it greedy: over long runs it can collapse into a poor local optimum and fall behind plain Random Search. This paper argues that the root cause is the evolution frequency, the number of training steps between selection rounds, and that no single frequency can simultaneously give fast adaptation and long-term robustness. To fix this, the paper introduces Multiple-Frequencies Population-Based Training (MF-PBT), which runs four sub-populations at different evolution frequencies and connects them with an asymmetric migration rule: when a fast agent is beaten by a slower one it is replaced entirely, while a slow agent beaten by a fast one imports only the fast agent's weights, not its hyperparameters. The paper reports that MF-PBT outperforms PBT, PB2, and Random Search on five Brax locomotion tasks at both 50 million and 1 billion steps, and that a large share of the gain comes from exploiting training variance rather than from hyperparameter mutation. If these results hold, MF-PBT gives practitioners a PBT variant that is less brittle to the choice of evolution frequency and keeps improving over very long training runs.

What carries the argument

The central mechanism is the evolution frequency—the number of training steps between evolutionary updates—together with the $\delta_i$ multipliers that make it heterogeneous. Each sub-population runs ordinary PBT at its own frequency; the fastest ($\delta=1$) provides agile anytime performance, the slowest ($\delta=50$) provides a long-horizon reference. The asymmetric migration rule does the linking: within an updating sub-population, the third quartile of agents is compared pairwise against the best external agents; a beaten agent facing a slower contender is replaced wholesale (weights and hyperparameters), while a beaten agent facing a faster contender receives only the weights. That asymmetry is what prevents greedy hyperparameters from leaking into the steady sub-populations while still allowing fast progress to propagate.

What would settle it

Reproduce the Humanoid experiment with the same seeds and budgets while logging the fitness of every $\delta=50$ sub-population agent at each 50-million-step checkpoint and again at 1 billion steps; if MF-PBT does not beat random search at the final horizon, or if the checkpoint-to-final ranking correlation is near zero, the claim that steady sub-populations supply a reliable long-term signal is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that PBT's long-run collapse has an identifiable mechanism: frequent evolution selects agents on short-term fitness, causing hyperparameter collapse (learning rates and exploration costs decay into a local optimum) and diversity collapse (repeated cloning shrinks the population into one basin of attraction). MF-PBT treats evolution frequency as the control variable, arranging agents into sub-populations with multipliers $\delta \in \{1, 10, 25, 50\}$ of the base interval $t_{\mathrm{ready}}$, and uses migration to let the slower sub-populations act as a check on greediness. The paper's empirical claim is that this design outperforms PBT, PB2, and Random Search at both the 50-million-step and 1-billion-step horizons across the Brax suite, and that ablations confirm both the multiplicity of frequencies and the asymmetric migration are necessary for the gain. In a separate set of experiments with hyperparameters fixed and only weights cloned, MF-PBT still improves performance, indicating variance exploitation is a real component of the gains.

Load-bearing premise

The method assumes that a sub-population that evolves only once every 50 million training steps produces a trustworthy ranking of long-term potential; if slow evolution is still too greedy or too sparse to be informative, the migration step is copying noise.

Editorial extensions

If this is right

  • On the reported Brax tasks, MF-PBT's best agent reaches higher rewards than PBT, PB2, and Random Search at both 50 million and 1 billion steps, so practitioners can use it for long-horizon RL without the usual collapse worry.
  • The ablation with different single frequencies shows the best $\delta$ varies by environment; MF-PBT's performance at least matches its best sub-population, making the frequency choice less brittle.
  • Variance-exploitation experiments imply that even without mutating hyperparameters, a well-designed cloning and migration scheme can beat both non-evolutive population baselines and standard PBT.
  • The finding that faster and slower frequencies trade off sample efficiency against long-term quality suggests other population-based HPO methods should treat selection interval as an explicit design axis, not a fixed default.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension beyond the paper is to adapt the frequency ladder online: if the slowest sub-population's rankings become uninformative, the algorithm could spend more agents at intermediate frequencies, or shorten $\delta_4$ as training progresses.
  • The asymmetry principle could be transplanted to any evolutionary optimizer where a conservative sub-population is meant to resist greedy transfer, for example in federated or multi-task settings importing only weights from fast clients and full configurations from slow ones.
  • The reported variance-exploitation gain suggests a cheap baseline for future RL HPO papers: a moderate population with fixed hyperparameters and weight-only cloning, which may already beat single-agent training.
  • Since the paper did not tune the $\delta$ values, a systematic sweep or a theoretical account of how $\delta$ interacts with environment horizon and population size would be a direct next step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Multiple-Frequencies Population-Based Training (MF-PBT), an extension of PBT that trains several sub-populations at different evolution frequencies and uses an asymmetric migration rule to transfer weights and/or hyperparameters between sub-populations. The motivation is that a single evolution frequency makes PBT greedy: frequent selection causes diversity collapse and hyperparameter collapse, while rare evolution sacrifices sample efficiency. MF-PBT is evaluated on five Brax environments with PPO, comparing against PBT, PB2, and random search, and is also studied in a variance-exploitation regime where hyperparameters are fixed. The paper reports that MF-PBT outperforms all baselines at 50M and 1B steps, and includes ablations on frequency choice, symmetric versus asymmetric migration, population size, and a backtracking baseline.

Significance. If the empirical claims hold, MF-PBT is a practically useful and simple modification of PBT that addresses a real failure mode, and the study of evolution frequency as a source of PBT greediness is a useful contribution. The paper provides public code, uses seven seeds with IQM/IQR reporting, and evaluates on multiple environments, which are commendable. The variance-exploitation experiments are an interesting addition. However, the central claim that MF-PBT 'consistently outperforms' all baselines at both horizons is supported only by point estimates without paired significance tests, and the mechanism--that the n=8 steady sub-population provides a reliable long-term signal--is not directly validated. The paper also makes an unsupported universal statement that MF-PBT will always perform at least as well as its best sub-population. These issues are fixable with additional analysis and controlled experiments, so the contribution is promising but not yet fully established.

major comments (4)
  1. [§4.1, Table 1, Figure 1] The sentence 'MF-PBT consistently outperforms PBT, PB2, and RS at both training horizons' is asserted from IQM/IQR point estimates only. IQM is a robust aggregate, but with seven seeds and no paired comparisons the word 'consistently' is stronger than the evidence. Several 50M-step differences are small (e.g., Hopper 2579 vs 2542; HalfCheetah 5154 vs 4914), and the IQR bands in Figure 1 overlap at many points. Please add paired per-seed tests (e.g., Wilcoxon signed-rank tests or bootstrap confidence intervals on the IQM difference) and report effect sizes for each environment/horizon, or soften the claim accordingly.
  2. [§3.1–3.3, Algorithm 2, Figure 4] The mechanism relies on the steady sub-population being a reliable long-term evaluator, as stated in §3.1: 'the best proxy for long-term performance is long-term performance itself.' However, the only evidence that δ=50 PBT is a useful steady signal is Figure 4, which runs 32-agent PBT. Inside MF-PBT the steady sub-population has n=8 agents (§3.3), and no experiment tests 8-agent PBT at δ=50 or otherwise verifies that the migrating n=8 agents actually encode a long-term signal rather than lucky random draws. Because Algorithm 2 lets a full clone from a steadier sub-population overwrite a dynamic agent, this is the core novelty. Please add a controlled experiment (e.g., compare MF-PBT with migration from an n=8 steady PBT arm versus an n=8 random-search arm, or report the trajectories and fitness of the steady agents) to support the claimed mechanism.
  3. [§5.1] The statement 'MF-PBT will always perform at least as well as its best-performing sub-population' is a universal claim unsupported by five Brax environments and seven seeds. It also does not follow from the algorithm's design, since migration can in principle disrupt a sub-population, and no formal argument or bound is provided. Please replace this with an empirical statement limited to the tested configurations, or provide a theoretical justification and conditions under which the property holds.
  4. [Appendix B.1] The comparison between the chosen δ configuration and the geometric-progression alternative changes both the δ values and tready simultaneously (tready=1e6 with δ=(1,10,25,50) versus tready=6e6 with δ=(1,2,4,8)). This confound means the reported advantage on Humanoid cannot be attributed to the spread of frequencies, and the experiment does not isolate the effect of δ. Please re-run with matched tready values, or clearly state that the comparison varies two factors and discuss the implications for the choice of δ.
minor comments (4)
  1. [§4.1] Please clarify the population size used for the PB2 baseline. The text notes that PB2 is designed for small populations such as N=8 but does not state explicitly whether PB2 was run with N=8 or N=32; if it was run with N=32, the comparison should be framed as a same-compute comparison, and if with N=8, the compute budget differs from the other methods.
  2. [§2.1, Appendix headings] There are several typos: 'imrpoves' in §2.1, 'Additionnal' in Appendix C, 'envisionned' in the supplementary introduction, and 'environnement' in Appendix C.2. These should be corrected.
  3. [Figure 2] The caption of Figure 2b would benefit from specifying how the 'best agent' history is traced and how the colors in the schedule map to the four δ values; currently the color legend is not described in the text.
  4. [§A.2] The discussion of selection rate as an alternative to evolution frequency is interesting, but the paper does not report an experiment varying the selection rate. Adding a sentence explaining why such an experiment was not run, or moving this discussion to future work, would make the scope clearer.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: MF-PBT's gains are empirical benchmark results against external baselines; δ and N selection is ordinary hyperparameter choice, not a fitted prediction.

full rationale

This is an empirical method paper whose central claim — 'MF-PBT consistently outperforms PBT, PB2, and RS at both training horizons' — is supported by direct benchmark comparisons against external baselines using reported simulation outcomes. The algorithm is defined constructively in Algorithm 1 and Algorithm 2: sub-populations evolve at distinct frequencies δi, and an asymmetric migration rule transfers weights and hyperparameters based on fitness comparisons. There is no equation in which the reported performance is constructed from a fitted quantity; the choices of δ-values, N, and tready are described in Section 4 as preliminary experimental selections, which is standard hyperparameter selection rather than a prediction derived from itself. The design principle 'the best proxy for long-term performance is long-term performance itself' (Section 3.1) is an algorithmic motivation, not a hidden restatement of the claimed result. The paper contains no load-bearing self-citations, no imported uniqueness theorem, and no renaming of a known empirical pattern. Potential concerns about statistical significance (point estimates without paired tests) and about tuning δ on the same benchmark environments are correctness and generalization risks, not circularity, so no circular step meets the evidentiary bar required by the analysis.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on a handful of judgment calls: the choice of sub-population frequencies and population size, and the assumption that slow evolution yields an unbiased view of long-term performance. The method itself is an algorithm, not a new physical entity; no new particles, forces, or conserved quantities are introduced.

free parameters (3)
  • Sub-population evolution frequencies δ = [1, 10, 25, 50]
    Chosen based on preliminary experiments on Humanoid (Appendix B.1) and compute budget; the paper reports that this spread-out configuration outperformed a geometric progression, and that δ4=50 was set so the slowest sub-population would have about 20 evolution steps in a 1B-step run.
  • Population size N = 32
    Selected from preliminary Humanoid experiments comparing N=16, 32, and 64 (Appendix B.2); the authors report diminishing returns beyond 32.
  • Number of sub-populations M = 4
    Set implicitly by N=32 and n=8 agents per sub-population; the paper states M=4 was held fixed without a dedicated ablation.
assumptions (3)
  • domain assumption The best proxy for long-term performance is long-term performance itself.
    Stated in Section 3.1; the method relies on slow sub-populations to identify long-term winners, rather than predicting them from learning curves as FIRE PBT does.
  • domain assumption Dynamic (high-frequency) sub-populations over-optimize local optima, while steady (low-frequency) sub-populations remain reliable enough to correct them.
    Core mechanism in Sections 3.1 and 3.2; if slow PBT were as greedy as fast PBT, the migration process would not provide a trustworthy corrective signal.
  • ad hoc to paper Importing only weights (not hyperparameters) from dynamic to steady agents protects the steady sub-population from hyperparameter collapse.
    Design choice in Algorithm 2 and Section 3.2; the paper asserts this protects the steady population but provides no isolated ablation of weight-only vs full transfer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multiple-Frequencies Population-Based Training." pith.science (2026). https://pith.science/paper/P4U7X2YU

@misc{pith2026250603225,
  author       = {Pith},
  title        = {Pith review of: Multiple-Frequencies Population-Based Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P4U7X2YU}},
  note         = {Machine review of arXiv:2506.03225}
}
read the original abstract

Reinforcement Learning's high sensitivity to hyperparameters is a source of instability and inefficiency, creating significant challenges for practitioners. Hyperparameter Optimization (HPO) algorithms have been developed to address this issue, among them Population-Based Training (PBT) stands out for its ability to generate hyperparameters schedules instead of fixed configurations. PBT trains a population of agents, each with its own hyperparameters, frequently ranking them and replacing the worst performers with mutations of the best agents. These intermediate selection steps can cause PBT to focus on short-term improvements, leading it to get stuck in local optima and eventually fall behind vanilla Random Search over longer timescales. This paper studies how this greediness issue is connected to the choice of evolution frequency, the rate at which the selection is done. We propose Multiple-Frequencies Population-Based Training (MF-PBT), a novel HPO algorithm that addresses greediness by employing sub-populations, each evolving at distinct frequencies. MF-PBT introduces a migration process to transfer information between sub-populations, with an asymmetric design to balance short and long-term optimization. Extensive experiments on the Brax suite demonstrate that MF-PBT improves sample efficiency and long-term performance, even without actually tuning hyperparameters.

Figures

Figures reproduced from arXiv: 2506.03225 by the authors.

Figure 1
Figure 1. Performance of MF-PBT, PB2, PBT, and RS on Brax environments. IQM across seven seeds, with IQR shaded. The performance of each algorithm is determined by the highest fitness score (mean evaluation reward over 512 episodes) among the 32 agents, evaluated every tready training steps. every tready “ 106 steps causes it to collapse into a poor optimum, while RS, which does not evolve, finds better solutions. Additionall… view at source ↗
Figure 2
Figure 2. Example of learning rate schedules for MF-PBT and PBT on the Humanoid environment. (a) MF-PBT snapshots at 750 million, 1.5 billion, and 3 billion training steps. Colors represent the sub-populations contribution to the schedule, showing how MF-PBT integrates input from various frequencies. (b) Comparison of the two final schedules, illustrating a case of hyperparameter collapse in PBT. Figure 2b compares the final … view at source ↗
Figure 3
Figure 3. Comparative performance of MF-PBT, PBT and a non-evolutive baseline for variance￾exploitation. IQM across seven seeds, with IQR shaded. Building on our discussion on variance-exploitation in section 2.1, we designed experiments to evaluate MF-PBT’s ability to leverage stochasticity in training outcomes to improve performance, even without hyperparameter tuning. In these experiments, all agents are fixed to use the d… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Impact of the evolution frequency in PBT. IQM across seven seeds, with IQR shaded. The resulting trajectories plotted in [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Ablation on the asymmetric migration. IQM across seven seeds, with IQR shaded. We compare MF-PBT to an alternative version where hyperparameters are always transferred along with weights, regardless of the δ-values (Symmetric). To highlight the importance of migration,…
Figure 6
Figure 6. Figure 6: compares our chosen configuration (tready “ 106 , δ1 “ 1, δ2 “ 10, δ3 “ 25, δ4 “ 50) with an alternative setup using a geometric progression (tready “ 6ˆ106 , δ1 “ 1, δ2 “ 2, δ3 “ 4, δ4 “ 8). The goal of this comparison is to assess how the spread of δ-values impacts M…
Figure 7
Figure 7. Figure 7: Impact of the population size. IQM across five seeds, with IQR shaded. Experiments on the Humanoid environment. C Additionnal Experiments C.1 Increasing Population Size One solution to improve PBT’s performance can be to increase the population size. In Jaderberg et al…
Figure 8
Figure 8. Figure 8: Increasing population size. IQM across seven seeds, with IQR shaded. C.2 Backtracking Zhang et al. (2021) proposed to add a backtracking mechanism to PBT, to prevent it from catastrophic forgetting. The method, dubbed PBT-BT (PBT with backtracking), keeps track of the …
Figure 9
Figure 9. Figure 9: Comparative performance of PBT-BT. IQM across seven seeds, with IQR shaded. We implemented PBT-BT with N “ 32, Ne “ 16 and δ “ 50. The training curves in [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Performance of MF-PBT, PBT, and RS on Pusher. IQM across seven seeds, with IQR shaded [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 19 canonical work pages

  1. [1]

    Deep reinforcement learning at the edge of the statistical precipice

    Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron C Courville, and Marc Bellemare. Deep reinforcement learning at the edge of the statistical precipice. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp.\ 29304--29320. Curran Associates, Inc., 202...

    work page 2021

  2. [2]

    Dehb: Evolutionary hyberband for scalable, robust and efficient hyperparameter optimization

    Noor Awad, Neeratyoy Mallik, and Frank Hutter. Dehb: Evolutionary hyberband for scalable, robust and efficient hyperparameter optimization. In Zhi-Hua Zhou (ed.), Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21 , pp.\ 2147--2153. International Joint Conferences on Artificial Intelligence Organization, 8 202...

    doi:10.24963/ijcai.2021/296 2021

  3. [3]

    Agent57: Outperforming the A tari human benchmark

    Adri \`a Puigdom \`e nech Badia, Bilal Piot, Steven Kapturowski, Pablo Sprechmann, Alex Vitvitskyi, Zhaohan Daniel Guo, and Charles Blundell. Agent57: Outperforming the A tari human benchmark. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Researc...

    work page 2020

  4. [4]

    Random search for hyper-parameter optimization

    James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13 0 (10): 0 281--305, 2012. URL http://jmlr.org/papers/v13/bergstra12a.html

    work page 2012

  5. [5]

    Algorithms for hyper-parameter optimization

    James Bergstra, R\' e mi Bardenet, Yoshua Bengio, and Bal\' a zs K\' e gl. Algorithms for hyper-parameter optimization. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K.Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011. URL https://proceedings.neurips.cc/paper_files/paper/2011/fil...

    2011

  6. [6]

    JAX : composable transformations of P ython+ N um P y programs, 2018

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake Vander P las, Skye Wanderman- M ilne, and Qiao Zhang. JAX : composable transformations of P ython+ N um P y programs, 2018. URL http://github.com/google/jax

    2018

  7. [7]

    Robust autonomy emerges from self-play, 2025

    Marco Cusumano-Towner, David Hafner, Alex Hertzberg, Brody Huval, Aleksei Petrenko, Eugene Vinitsky, Erik Wijmans, Taylor Killian, Stuart Bowers, Ozan Sener, Philipp Krähenbühl, and Vladlen Koltun. Robust autonomy emerges from self-play, 2025. URL https://arxiv.org/abs/2502.03349

    arXiv 2025

  8. [8]

    Faster improvement rate population based training

    Valentin Dalibard and Max Jaderberg. Faster improvement rate population based training. CoRR, abs/2109.13800, 2021. URL https://arxiv.org/abs/2109.13800

    arXiv 2021

Show all 32 references
  1. [9]

    Hyperparameters in reinforcement learning and how to tune them

    Theresa Eimer, Marius Lindauer, and Roberta Raileanu. Hyperparameters in reinforcement learning and how to tune them. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Confere...

    2023

  2. [10]

    BOHB : Robust and efficient hyperparameter optimization at scale

    Stefan Falkner, Aaron Klein, and Frank Hutter. BOHB : Robust and efficient hyperparameter optimization at scale. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Researc...

    2018

  3. [11]

    Hyperparameter optimization

    Matthias Feurer and Frank Hutter. Hyperparameter optimization. In Automatic Machine Learning: Methods, Systems, Challenges, pp.\ 3--38. Springer, 2019

    2019

  4. [12]

    Model-agnostic meta-learning for fast adaptation of deep networks

    Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Resear...

    2017

  5. [13]

    Franke, Gregor Koehler, Andr \'e Biedenkapp, and Frank Hutter

    J \"o rg K.H. Franke, Gregor Koehler, Andr \'e Biedenkapp, and Frank Hutter. Sample-efficient automated deep reinforcement learning. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=hSjxQ3B7GWq

    2021

  6. [14]

    Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem

    C. Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem. Brax -- a differentiable physics engine for large scale rigid body simulation, 2021

    2021

  7. [15]

    Deep reinforcement learning that matters

    Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, and David Meger. Deep reinforcement learning that matters. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligen...

    2018

  8. [16]

    Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, Chrisantha Fernando, and Koray Kavukcuoglu

    Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M. Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, Chrisantha Fernando, and Koray Kavukcuoglu. Population based training of neural networks. CoRR, abs/1711.09846, 2017. URL ...

    2017 arXiv

  9. [17]

    Hyperband: A novel bandit-based approach to hyperparameter optimization

    Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. Hyperband: A novel bandit-based approach to hyperparameter optimization. Journal of Machine Learning Research, 18 0 (185): 0 1--52, 2018. URL http://jmlr.org/papers/v18/16-558.html

    2018

  10. [18]

    Siqi Liu, Guy Lever, Zhe Wang, Josh Merel, S. M. Ali Eslami, Daniel Hennes, Wojciech M. Czarnecki, Yuval Tassa, Shayegan Omidshafiei, Abbas Abdolmaleki, Noah Y. Siegel, Leonard Hasenclever, Luke Marris, Saran Tunyasuvunakool, H. Francis Song, Markus Wulfmeier, Paul Muller, Tuo...

    2022 doi

  11. [19]

    Provably efficient online hyperparameter optimization with population-based bandits

    Jack Parker-Holder, Vu Nguyen, and Stephen J Roberts. Provably efficient online hyperparameter optimization with population-based bandits. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp...

    2020

  12. [20]

    Tuning mixed input hyperparameters on the fly for efficient population based autorl

    Jack Parker-Holder, Vu Nguyen, Shaan Desai, and Stephen J Roberts. Tuning mixed input hyperparameters on the fly for efficient population based autorl. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processi...

    2021

  13. [21]

    Automated reinforcement learning (autorl): A survey and open problems

    Jack Parker-Holder, Raghu Rajan, Xingyou Song, André Biedenkapp, Yingjie Miao, Theresa Eimer, Baohe Zhang, Vu Nguyen, Roberto Calandra, Aleksandra Faust, Frank Hutter, and Marius Lindauer. Automated reinforcement learning (autorl): A survey and open problems. Journal of Artifi...

    2022 doi

  14. [22]

    Rusu, Sergio Gomez Colmenarejo, C aglar G \" u l c ehre, Guillaume Desjardins, James Kirkpatrick, Razvan Pascanu, Volodymyr Mnih, Koray Kavukcuoglu, and Raia Hadsell

    Andrei A. Rusu, Sergio Gomez Colmenarejo, C aglar G \" u l c ehre, Guillaume Desjardins, James Kirkpatrick, Razvan Pascanu, Volodymyr Mnih, Koray Kavukcuoglu, and Raia Hadsell. Policy distillation. In Yoshua Bengio and Yann LeCun (eds.), 4th International Conference on Learnin...

    2016 arXiv

  15. [23]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. CoRR, abs/1707.06347, 2017. URL http://dblp.uni-trier.de/db/journals/corr/corr1707.html#SchulmanWDRK17

    2017 arXiv

  16. [24]

    Ofer M. Shir. Niching in Evolutionary Algorithms, pp.\ 1035--1069. Springer Berlin Heidelberg, Berlin, Heidelberg, 2012. ISBN 978-3-540-92910-9. doi:10.1007/978-3-540-92910-9_32. URL https://doi.org/10.1007/978-3-540-92910-9_32

    2012 doi

  17. [25]

    Leslie N. Smith. Cyclical learning rates for training neural networks. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp.\ 464--472, 2017. doi:10.1109/WACV.2017.58

    2017 doi

  18. [26]

    Francis Song, Abbas Abdolmaleki, Jost Tobias Springenberg, Aidan Clark, Hubert Soyer, Jack W

    H. Francis Song, Abbas Abdolmaleki, Jost Tobias Springenberg, Aidan Clark, Hubert Soyer, Jack W. Rae, Seb Noury, Arun Ahuja, Siqi Liu, Dhruva Tirumala, Nicolas Heess, Dan Belov, Martin A. Riedmiller, and Matthew M. Botvinick. V-MPO: on-policy maximum a posteriori policy optimi...

    1909 arXiv

  19. [27]

    Adapting Crossover in Evolutionary Algorithms

    William Michael Spears. Adapting Crossover in Evolutionary Algorithms . In Evolutionary Programming IV: Proceedings of the Fourth Annual Conference on Evolutionary Programming . The MIT Press, 08 1995. ISBN 9780262290920. doi:10.7551/mitpress/2887.003.0035. URL https://doi.org...

    1995 doi

  20. [28]

    Sutton and Andrew G

    Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. The MIT Press, second edition, 2018. URL http://incompleteideas.net/book/the-book-2nd.html

    2018

  21. [29]

    Ball, Vu Nguyen, Binxin Ru, and Michael Osborne

    Xingchen Wan, Cong Lu, Jack Parker-Holder, Philip J. Ball, Vu Nguyen, Binxin Ru, and Michael Osborne. Bayesian generational population-based training. In Isabelle Guyon, Marius Lindauer, Mihaela van der Schaar, Frank Hutter, and Roman Garnett (eds.), Proceedings of the First I...

    2022

  22. [30]

    Meta-gradient reinforcement learning

    Zhongwen Xu, Hado P van Hasselt, and David Silver. Meta-gradient reinforcement learning. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. UR...

    2018

  23. [31]

    On the importance of hyperparameter optimization for model-based reinforcement learning

    Baohe Zhang, Raghu Rajan, Luis Pineda, Nathan Lambert, Andr \'e Biedenkapp, Kurtland Chua, Frank Hutter, and Roberto Calandra. On the importance of hyperparameter optimization for model-based reinforcement learning. In Arindam Banerjee and Kenji Fukumizu (eds.), Proceedings of...

    2021

  24. [32]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.