REVIEW 11 cited by
What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Study
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In recent years, on-policy reinforcement learning (RL) has been successfully applied to many different continuous control tasks. While RL algorithms are often conceptually simple, their state-of-the-art implementations take numerous low- and high-level design decisions that strongly affect the performance of the resulting agents. Those choices are usually not extensively discussed in the literature, leading to discrepancy between published descriptions of algorithms and their implementations. This makes it hard to attribute progress in RL and slows down overall progress [Engstrom'20]. As a step towards filling that gap, we implement >50 such ``choices'' in a unified on-policy RL framework, allowing us to investigate their impact in a large-scale empirical study. We train over 250'000 agents in five continuous control environments of different complexity and provide insights and practical recommendations for on-policy training of RL agents.
Forward citations
Cited by 11 Pith papers
-
MuJoCo Playground
An open-source, MJX-based robot learning framework with integrated batch rendering that provides fast training and demonstrates sim-to-real transfer on six robot platforms.
-
The Importance of Encoder Choice:A Tabular-Image Study
Tabular encoder choice reorders multimodal rankings, can erase apparent fusion gains, and requires non-vanilla extraction for in-context learning models to avoid train-test representation shift.
-
Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Control
Logic-gate circuits trained with gradient descent can match neural-network policies on most MuJoCo continuous-control tasks and run on FPGAs in a few clock cycles.
-
Online Training and Pruning of Deep Reinforcement Learning Networks
A method that prunes OFENet-based reinforcement learning networks during training, reducing them to a fraction of their original size with minimal performance loss.
-
Learning human-to-robot handovers through 3D scene reconstruction
A handover policy trained only on images rendered from a sparse-view Gaussian Splatting scene can deploy on a real robot without real-robot training data.
-
On the Effect of Regularization in Policy Mirror Descent
A large empirical sweep shows that in Policy Mirror Descent, MDP and Drift regularizers are partly substitutable, yet their precise combination determines temperature robustness.
-
Magistral
Pure RL alone, without distilled reasoning traces, turned Mistral's base models into strong reasoning models on math and coding benchmarks.
-
Simulating Errors in Touchscreen Typing
Typoist is a simulation model that reproduces slips, lapses, and mistakes in touchscreen typing, with error rates matching human data in a new benchmark.
-
Preventing Learning Stagnation in PPO by Scaling to 1 Million Parallel Environments
PPO plateaus can be avoided by increasing the number of parallel environments, which reduces both the outer-loop step size and update noise; scaling to 1M environments sustained improvement to 1T transitions.
-
RedRFT: A Light-Weight Benchmark for Reinforcement Fine-Tuning-Based Red Teaming
RedRFT is a new open-source benchmark with a unified PPO backbone, five reimplemented red teaming baselines, a proposed diversity metric, and ablation insights.
-
Communicating Smartly in Molecular Communication Environments: Neural Networks in the Internet of Bio-Nano Things
A broad, code-augmented survey of neural network methods for molecular communication in the Internet of Bio-Nano Things, including a dataset accessibility audit and open challenges.
Discussion (0). Continue with ORCID to comment.