REVIEW 7 cited by
SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent advances in CV and NLP have been largely driven by scaling up the number of network parameters, despite traditional theories suggesting that larger networks are prone to overfitting. These large networks avoid overfitting by integrating components that induce a simplicity bias, guiding models toward simple and generalizable solutions. However, in deep RL, designing and scaling up networks have been less explored. Motivated by this opportunity, we present SimBa, an architecture designed to scale up parameters in deep RL by injecting a simplicity bias. SimBa consists of three components: (i) an observation normalization layer that standardizes inputs with running statistics, (ii) a residual feedforward block to provide a linear pathway from the input to output, and (iii) a layer normalization to control feature magnitudes. By scaling up parameters with SimBa, the sample efficiency of various deep RL algorithms-including off-policy, on-policy, and unsupervised methods-is consistently improved. Moreover, solely by integrating SimBa architecture into SAC, it matches or surpasses state-of-the-art deep RL methods with high computational efficiency across DMC, MyoSuite, and HumanoidBench. These results demonstrate SimBa's broad applicability and effectiveness across diverse RL algorithms and environments.
Forward citations
Cited by 7 Pith papers
-
Balancing Expressivity and Robustness: Constrained Rational Activations for Reinforcement Learning
A constrained rational activation with denominator degree one larger than numerator and no constant term stabilizes high-UTD continuous control, while trading off long-term plasticity.
-
On the Effect of Regularization in Policy Mirror Descent
A large empirical sweep shows that in Policy Mirror Descent, MDP and Drift regularizers are partly substitutable, yet their precise combination determines temperature robustness.
-
Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learners
A single multi-task RL agent using a large regularized critic, categorical value loss, and task embeddings achieves state-of-the-art results across 283 tasks and transfers efficiently to new tasks.
-
Hadamax Encoding: Elevating Performance in Model-Free Atari
Hadamax, a Hadamard-product and max-pooling encoder, improves PQN's median human-normalized Atari-57 score by about 80% with no algorithmic changes.
-
RN-D: Discretized Categorical Actors for On-Policy Reinforcement Learning
A discretized categorical actor paired with a pre-LayerNorm residual MLP (RN-D) improves PPO's final performance and sample efficiency on continuous-control benchmarks.
-
A Forget-and-Grow Strategy for Deep Reinforcement Learning Scaling in Continuous Control
Forget and Grow (FoG) combines decaying replay weights for old experiences with progressive critic-network expansion to improve continuous-control reinforcement learning, beating BRO, SimBa, and TD-MPC2 on most of 41 ...
-
Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategies
A survey that categorizes deep reinforcement learning scaling strategies into data, network, and training budget dimensions and outlines challenges for scaling DRL systems.
Discussion (0). Sign in to comment.