Pith. sign in

REVIEW 5 major objections 5 minor 25 references

Evolutionary Policy Optimization

T0 review · 5 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper shows that alternating neuroevolution with PPO improves both sample efficiency and policy quality on Atari benchmarks.

desk verdict EPO is a clean hybrid of PPO and neuroevolution, but its sample-efficiency claims are confounded by wall-clock-capped comparisons and in-sample tuning. read the letter →

arxiv 2504.12568 v1 pith:NZJCXNQE submitted 2025-04-17 cs.LG cs.NE

Zelal Su "Lain" Mustafaoglu , Keshav Pingali , Risto Miikkulainen This is my paper · ORCID
classification cs.LGcs.NE
keywords evolutionarypolicyoptimizationneuroevolutionproximalexplorationvsexploitationsampleefficiencyAtarireinforcementlearningcrossovermutation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Evolutionary Policy Optimization (EPO) is a hybrid algorithm that alternates between a policy gradient method (PPO) for fine-grained local optimization and neuroevolution for global exploration. The paper's central claim is that this combination outperforms either approach alone: on Breakout, EPO reduces sample count by 26.8% compared with PPO and by 57.3% compared with pure evolution, and on Pong, EPO's best policy reaches a reward of 11.0 whereas PPO's best reaches 1.0. EPO uses PPO to pre-train a population, then applies elitism, fitness-weighted crossover, adaptive mutation, and a short PPO fine-tuning step to offspring. The practical stakes are that RL systems can get both exploration and exploitation without a strict sample-efficiency penalty, at least on tasks that mix dense and sparse reward structure.

What carries the argument

The central object is the evolutionary loop itself, with three operations: elitism (carrying the top $\mathrm{E}$ agents forward), fitness-weighted crossover of neural network weight vectors $c = \alpha p_1 + (1-\alpha)p_2$ where $\alpha = f_1/(f_1+f_2+\epsilon)$, and adaptive Gaussian mutation whose scale grows as parent fitnesses converge. The loop's variation is carried by the crossover and mutation, while PPO acts as the exploitation engine by pre-training the initial clone population and by locally refining offspring; this alternating structure is what lets global search and gradient descent reinforce each other.

What would settle it

A direct test would measure the average reward of crossover offspring immediately after crossover and after their 500-step PPO fine-tuning on Breakout. If the fine-tuned crossover offspring are not better, on average, than offspring produced by mutating a single elite, then the crossover operation is not carrying EPO's gains and the algorithm would perform just as well without it. A second check: recompute sample counts including the overhead of fitness evaluations under the same wall-clock budget; if EPO's sample-efficiency advantage over PPO disappears, the claim of 26.8% improvement is an artifact of how samples are counted.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that neuroevolution and policy gradients need not be competing alternatives. By cloning a PPO-pretrained policy into a population, evolving the population through fitness-weighted weight crossover and adaptive Gaussian mutation, and fine-tuning unmutated offspring with PPO for 500 timesteps, EPO discovers policies that plain PPO does not find and uses fewer environment samples to reach comparable reward on Breakout. The paper reports best-reward 11.0 on Pong versus 1.0 for PPO and a 26.8% sample-count reduction on Breakout over PPO. These results are the evidence for the claim that the hybrid is effective for tasks requiring both exploration and local optimization.

Load-bearing premise

The load-bearing assumption is that interpolating the weights of two trained policies, then running PPO for 500 timesteps, yields offspring worth keeping; if weight averaging routinely destroys the policy structure that PPO cannot repair, the evolutionary loop stops producing useful variation.

Editorial extensions

If this is right

  • On tasks with sparse or deceptive rewards, EPO should reach a given reward level with about a quarter fewer environment interactions than PPO alone, based on the Breakout results.
  • EPO's best-found policies can be qualitatively better than PPO's best within the same wall-clock budget, as shown by the Pong best-reward gap (11.0 vs. 1.0).
  • A short fine-tuning budget of 500 PPO timesteps is enough to make crossover offspring useful, so the hybrid's sample overhead per generation is modest.
  • Transferring a policy from a similar task (Pong to Breakout) does not accelerate EPO, so the gains come from within-task evolution and not from cross-task reuse.
  • Hyperparameters tuned on Breakout (mutation probability 0.3, population size 8, elite count 3) transferred to Pong without re-tuning, suggesting the method is not highly sensitive to task identity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If fitness-weighted weight averaging is the crucial variation operator, then EPO's success might be fragile when network architectures exhibit permutation symmetry, where interpolating weights can map to functionally meaningless policies; testing crossover in a permutation-invariant representation would separate this effect.
  • The sample-count comparison on Pong favors PPO, so the sweeping claim of 'sample efficiency' is a per-task phenomenon; a more careful comparison would track the full reward–sample Pareto frontier rather than a single metric.
  • One testable prediction follows from the mutation-scaling rule: when parent fitnesses are nearly equal, mutation noise increases, so EPO should explore most aggressively when the population is undecided about direction; a diagnostic would record mutation scale over training and correlate it with reward jumps.
  • The 500-step fine-tuning budget is a free parameter of the hybrid; on harder or higher-dimensional tasks, the required budget likely grows with task complexity, and the paper does not establish a scaling law.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Evolutionary Policy Optimization (EPO), a hybrid algorithm that combines neuroevolution (elitism, fitness-weighted crossover, mutation) with PPO pre-training and fine-tuning. EPO is evaluated on Atari Pong and Breakout against PPO and a pure evolution baseline, with the central claim that EPO improves both policy quality and sample efficiency. The paper also reports an ablation study and a transfer-learning experiment. The algorithm is clearly described, but the empirical evidence for the central claim is undermined by methodological flaws in the sample-efficiency comparison, in-sample hyperparameter tuning, and weak statistical support.

Significance. If the central claim were established, EPO would be a useful contribution: a simple integration of neuroevolution and policy gradients could improve exploration and exploit local optimization, potentially benefiting sparse-reward and exploration-heavy RL tasks. The paper provides a clear algorithmic description, an ablation study, and experiments on standard Atari benchmarks. However, the reported results do not currently support the claim. The sample-efficiency comparison is confounded with wall-clock time, the Breakout result is in-sample because hyperparameters were tuned on that task, and the policy-quality improvements are within one standard deviation or rely on an extreme-order statistic. These issues affect the paper's main conclusions, so the contribution is not yet validated.

major comments (5)
  1. [Section 5.1, Table 4] The sample-efficiency comparison is confounded with wall-clock time. All methods are trained for a fixed wall-clock duration (7,200 seconds for Breakout, 10,000 seconds for Pong), and then the total sample count is reported as a measure of sample efficiency. A lower sample count at fixed wall-clock time does not imply better sample efficiency; it may reflect lower throughput. Indeed, pure evolution consumes 1.92e6 samples in the same wall-clock budget as EPO's 0.82e6, likely because it runs more environment interactions per second, not because EPO is more sample-efficient. To support the claims of a 26.8% and 57.3% sample-efficiency improvement, the authors must compare reward at matched sample budgets or plot reward versus samples; the current results do not measure sample efficiency.
  2. [Section 4.3, Table 4] The main Breakout result is measured on the same task used for hyperparameter tuning. Hyperparameters (mutation probability, elite count, population size) were optimized over 75 trials of 240 seconds of training on Breakout, and the Breakout results in Table 4 are then reported for this configuration. This makes the headline sample-efficiency and reward numbers in-sample. The out-of-sample check on Pong is not reported with any numeric detail; the paper only states that 'EPO with the identified hyperparameters consistently performed well' over 20 trials, with no results shown. The generalization claim is therefore unsupported, and the Breakout improvements may reflect tuning to that specific environment.
  3. [Tables 3 and 4] The statistical evidence for 'better policy quality' is weak. On Pong, the mean rewards are -6.25 +/- 3.42 for EPO versus -8.03 +/- 5.17 for PPO; these overlap at one standard deviation. The 'best reward' of 11.0 versus 1.0 is an extreme-order statistic with no significance test and is not a reliable measure of policy quality. On Breakout, mean rewards are 3.15 +/- 1.45 versus 3.02 +/- 1.43, again overlapping. No significance tests, confidence intervals for the differences, or effect sizes are provided, so the claim that EPO 'discovers significantly better policies' is not supported by the data.
  4. [Section 5.2, Pong results] The paper's own Pong results contradict the abstract's claim that EPO improves both policy quality and sample efficiency. Table 3 shows EPO uses 2.36e6 samples versus PPO's 1.58e6 samples at the same wall-clock budget, and the text acknowledges 'EPO sacrifices some sample efficiency but gains a substantial improvement in policy quality.' This direct contradiction means the central claim as stated in the abstract and introduction cannot stand without substantial re-framing.
  5. [Section 4.1.2, Algorithm 1] The core evolutionary operator—fitness-weighted averaging of neural network weights—is not validated beyond a single sentence. The paper states that 'other crossover techniques, such as random parameter masking, were tested but did not outperform the proposed weighted averaging approach,' but no data or analysis is provided. The assumption that interpolating weights of deep CNNs produces viable offspring that 500 timesteps of PPO fine-tuning can repair is a load-bearing assumption of the algorithm, yet it is not tested in the ablation studies. This leaves a correctness-risk concern about whether the evolutionary search actually explores effectively.
minor comments (5)
  1. [Section 3] The text contains the placeholder '[NEED TO ADD CITATIONS]' in the Previous Work section, which indicates the manuscript is incomplete and needs a careful revision.
  2. [Figure 3] The caption and axis labels use abbreviations 'EPO-TL' and 'EPO-NoPT' without defining them in the figure; the text introduces these names only later in Section 6 or uses different terms, which hurts readability.
  3. [Section 5.1] The sentence 'the computational overhead of pre-processing these visual inputs makes Atari a suitable benchmark for evaluating sample complexity' is confusing; preprocessing overhead is unrelated to sample complexity, and the point is not developed.
  4. [Section 4.3 and Table 6] In the appendix table, the phrase 'with with' appears in the title, and it would be helpful to report the standard deviations across runs for the hyperparameter-search rewards, not just the mean.
  5. [General] The 'pure evolution' baseline is not fully described: its mutation rate, population size, and other hyperparameters are not given, making it hard to assess the fairness of the comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EPO's construction, claims, and baselines are not derived from the reported outcomes.

full rationale

The paper is an empirical algorithmic proposal rather than a derivation. EPO is defined by Algorithm 1 and Sections 4.1-4.2 (elitism, fitness-weighted crossover, adaptive Gaussian mutation, and PPO pre-training/fine-tuning) independently of the Atari results; no equation in the method section is defined in terms of the reported rewards or sample counts. The self-citations (refs [13], [5], [9], [19]) are provenance and background material, and they are not used to justify the central effectiveness claim; that claim rests on the experiments in Section 5. The in-sample hyperparameter tuning on Breakout (Section 4.3) is a methodological weakness that risks overfitting, and the wall-clock-capped sample counts in Table 4 conflate computational throughput with sample efficiency, but neither is a circular reduction: the optimized hyperparameters are not the measured quantities (sample counts), and the algorithm's operation does not encode the outcome. The Pong evaluation provides a partially out-of-sample check. Therefore no self-definitional, fitted-input-called-prediction, or self-citation-load-bearing step can be exhibited; the appropriate finding is no significant circularity.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The central claim rests on three tuned hyperparameters (mutation probability, elite count, population size) that were optimized on Breakout, two hand-picked training budgets (pre-training and fine-tuning steps), and an unstated proportionality constant for mutation noise. The crossover assumption is an ad hoc domain assumption that the paper does not stress-test.

free parameters (6)
  • mutation_probability = 0.3
    Tuned via Optuna Bayesian optimization over 75 trials on Breakout (Section 4.3).
  • elite_count = 3
    Tuned via Optuna on Breakout (Section 4.3).
  • population_size = 8
    Tuned via Optuna on Breakout (Section 4.3).
  • pre_training_steps = 30,000
    Chosen from ablation sweeping values (Section 5.3), not optimized by Optuna.
  • fine_tuning_steps = 500
    Set in an ablation varying 0 to 1000 (Section 5.3), not optimized.
  • mutation_noise_sigma_scale = unspecified
    The mutation noise variance is stated as 'proportional to the scaling_factor' (Section 4.1.3) without giving the proportionality constant; this hidden parameter is needed for exact reproduction.
assumptions (3)
  • standard math Standard MDP framework and policy gradient theorem
    Used in Section 2.1 as background, standard RL assumptions.
  • domain assumption Atari Pong and Breakout are representative of tasks requiring exploration-exploitation tradeoffs
    The paper selects only these two games and generalizes from them in Sections 5 and 8.
  • ad hoc to paper Fitness-weighted weight averaging produces viable offspring for future optimization
    Core crossover operation in Section 4.1.2; no theoretical or systematic empirical justification provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evolutionary Policy Optimization." pith.science (2026). https://pith.science/paper/NZJCXNQE

@misc{pith2026250412568,
  author       = {Pith},
  title        = {Pith review of: Evolutionary Policy Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NZJCXNQE}},
  note         = {Machine review of arXiv:2504.12568}
}
read the original abstract

A key challenge in reinforcement learning (RL) is managing the exploration-exploitation trade-off without sacrificing sample efficiency. Policy gradient (PG) methods excel in exploitation through fine-grained, gradient-based optimization but often struggle with exploration due to their focus on local search. In contrast, evolutionary computation (EC) methods excel in global exploration, but lack mechanisms for exploitation. To address these limitations, this paper proposes Evolutionary Policy Optimization (EPO), a hybrid algorithm that integrates neuroevolution with policy gradient methods for policy optimization. EPO leverages the exploration capabilities of EC and the exploitation strengths of PG, offering an efficient solution to the exploration-exploitation dilemma in RL. EPO is evaluated on the Atari Pong and Breakout benchmarks. Experimental results show that EPO improves both policy quality and sample efficiency compared to standard PG and EC methods, making it effective for tasks that require both exploration and local optimization.

Figures

Figures reproduced from arXiv: 2504.12568 by the authors.

Figure 1
Figure 1. Training rewards over 10,000 seconds of wall clock [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 3
Figure 3. Sample complexity on Breakout across methods [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 2
Figure 2. Training rewards over 7,200 seconds of wall clock [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Training rewards over 7200 seconds of wall clock [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 6
Figure 6. Figure 6: Training rewards over 7,200 seconds of wall clock [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 5
Figure 5. Figure 5: Training reward over 240 seconds across different [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 3 canonical work pages

  1. [1]

    Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Frame- work. CoRR abs/1907.10902 (2019). arXiv:1907.10902 http://arxiv.org/abs/1907. 10902

    arXiv 2019

  2. [2]

    Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling

    Marc G. Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. 2012. The Arcade Learning Environment: An Evaluation Platform for General Agents.CoRR abs/1207.4708 (2012). arXiv:1207.4708 http://arxiv.org/abs/1207.4708

    arXiv 2012

  3. [3]

    James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. Algorithms for Hyper-Parameter Optimization. In Advances in Neural Information Processing Systems, J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K.Q. Weinberger (Eds.), Vol. 24. Curran Associates, Inc. https://proceedings.neurips.cc/paper_ files/paper/2011/file/86e8f7ab32cfd125...

    work page 2011

  4. [4]

    Stanley, and Jeff Clune

    Edoardo Conti, Vashisht Madhavan, Felipe Petroski Such, Joel Lehman, Ken- neth O. Stanley, and Jeff Clune. 2017. Improving Exploration in Evolution Strate- gies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents. CoRR abs/1712.06560 (2017). arXiv:1712.06560 http://arxiv.org/abs/1712. 06560

    arXiv 2017

  5. [5]

    Olivier Francon, Santiago Gonzalez, Babak Hodjat, Elliot Meyerson, Risto Miikku- lainen, Xin Qiu, and Hormoz Shahrzad. 2020. Effective Reinforcement Learning through Evolutionary Surrogate-Assisted Prescription. CoRR abs/2002.05368 (2020). arXiv:2002.05368 https://arxiv.org/abs/2002.05368

    work page Pith review arXiv 2020

  6. [6]

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. 2018. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. CoRR abs/1801.01290 (2018). arXiv:1801.01290 http://arxiv. org/abs/1801.01290

    arXiv 2018

  7. [7]

    LeCun, B

    Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. 1989. Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation 1, 4 (1989), 541–551. https://doi.org/10.1162/neco.1989.1.4. 541

    doi:10.1162/neco.1989.1.4 1989

  8. [8]

    Joel Lehman and Kenneth O. Stanley. 2011. Abandoning Objectives: Evolution Through the Search for Novelty Alone. Evolutionary Computation 19, 2 (2011), 189–223. https://doi.org/10.1162/EVCO_a_00025

    doi:10.1162/evco_a_00025 2011

Show all 25 references
  1. [9]

    Risto Miikkulainen, Jason Zhi Liang, Elliot Meyerson, Aditya Rawal, Daniel Fink, Olivier Francon, Bala Raju, Hormoz Shahrzad, Arshak Navruzyan, Nigel Duffy, and Babak Hodjat. 2017. Evolving Deep Neural Networks. CoRR abs/1703.00548 (2017). arXiv:1703.00548 http://arxiv.org/abs...

    2017 arXiv

  2. [10]

    Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu

    Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timo- thy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asyn- chronous Methods for Deep Reinforcement Learning. CoRR abs/1602.01783 (2016). arXiv:1602.01783 http://arxiv.org/abs/1602.01783

    2016 arXiv

  3. [11]

    Riedmiller

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin A. Riedmiller. 2013. Playing Atari with Deep Reinforcement Learning. CoRR abs/1312.5602 (2013). arXiv:1312.5602 http://arxiv.org/abs/1312.5602

    2013 arXiv

  4. [12]

    Rusu, Joel Veness, Marc G

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin A. Riedmiller, Andreas Kirkeby Fidjeland, Georg Ostrovski, Stig Petersen, Charlie Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wi...

  5. [13]

    Zelal Su Mustafaoglu, Keshav Pingali, and Risto Miikkulainen. 2025. Evolu- tionary Policy Optimization. Poster at Genetic and Evolutionary Computation Conference (GECCO 2025)

    2025

  6. [14]

    Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernes- tus, and Noah Dormann. 2021. Stable-Baselines3: Reliable Reinforcement Learn- ing Implementations. Journal of Machine Learning Research 22, 268 (2021), 1–8. http://jmlr.org/papers/v22/20-1364.html

    2021

  7. [15]

    Reuven Rubinstein. 1999. The Cross-Entropy Method for Combinatorial and Continuous Optimization. 1, 2 (Sept. 1999), 127–190. https://doi.org/10.1023/A: 1010091220143

    1999 doi

  8. [16]

    Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. 2017. Evolution Strategies as a Scalable Alternative to Reinforcement Learning. arXiv:1703.03864 [stat.ML] https://arxiv.org/abs/1703.03864

    2017 arXiv

  9. [17]

    Jordan, and Pieter Abbeel

    John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, and Pieter Abbeel. 2015. Trust Region Policy Optimization. CoRR abs/1502.05477 (2015). arXiv:1502.05477 http://arxiv.org/abs/1502.05477

    2015 arXiv

  10. [18]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

  11. [19]

    Stanley and R

    K.O. Stanley and R. Miikkulainen. 2002. Efficient evolution of neural network topologies. In Proceedings of the 2002 Congress on Evolutionary Computation. CEC’02 (Cat. No.02TH8600), Vol. 2. 1757–1762 vol.2. https://doi.org/10.1109/CEC. 2002.1004508

    2002 arXiv

  12. [20]

    Stanley, and Jeff Clune

    Felipe Petroski Such, Vashisht Madhavan, Edoardo Conti, Joel Lehman, Ken- neth O. Stanley, and Jeff Clune. 2017. Deep Neuroevolution: Genetic Algo- rithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning. CoRR abs/1712.06567 (2017). ar...

    2017 arXiv

  13. [21]

    Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. 1999. Policy Gradient Methods for Reinforcement Learning with Function Approxima- tion. In Advances in Neural Information Processing Systems , S. Solla, T. Leen, and K. Müller (Eds.), Vol. 12. MIT Press. h...

    1999

  14. [22]

    Ziyu Wang, Victor Bapst, Nicolas Heess, Volodymyr Mnih, Rémi Munos, Koray Kavukcuoglu, and Nando de Freitas. 2016. Sample Efficient Actor-Critic with Experience Replay. CoRR abs/1611.01224 (2016). arXiv:1611.01224 http://arxiv. org/abs/1611.01224

    2016 arXiv

  15. [23]

    Williams

    Ronald J. Williams. 1992. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Mach. Learn. 8, 3–4 (May 1992), 229–256. https://doi.org/10.1007/BF00992696 A HYPERPARAMETER OPTIMIZATION Table 6 shows the values of a subset of the hyperparam...

    1992 doi

  16. [2015]

    Nature 518 (2015), 529–533

    Human-level control through deep reinforcement learning. Nature 518 (2015), 529–533. https://api.semanticscholar.org/CorpusID:205242740

    2015

  17. [2017]

    CoRR abs/1707.06347 (2017)

    Proximal Policy Optimization Algorithms. CoRR abs/1707.06347 (2017). arXiv:1707.06347 http://arxiv.org/abs/1707.06347

    2017 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.