Pith. sign in

REVIEW 5 cited by

CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicity

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1902.05605 v4 pith:J4RJKY2B submitted 2019-02-14 cs.LG stat.ML

Aditya Bhatt , Daniel Palenicek , Boris Belousov , Max Argus , Artemij Amiranashvili , Thomas Brox , Jan Peters This is my paper
classification cs.LGstat.ML
keywords sampleefficiencycrossqcomputationalcurrentbatchcostdeep
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Sample efficiency is a crucial problem in deep reinforcement learning. Recent algorithms, such as REDQ and DroQ, found a way to improve the sample efficiency by increasing the update-to-data (UTD) ratio to 20 gradient update steps on the critic per environment sample. However, this comes at the expense of a greatly increased computational cost. To reduce this computational burden, we introduce CrossQ: A lightweight algorithm for continuous control tasks that makes careful use of Batch Normalization and removes target networks to surpass the current state-of-the-art in sample efficiency while maintaining a low UTD ratio of 1. Notably, CrossQ does not rely on advanced bias-reduction schemes used in current methods. CrossQ's contributions are threefold: (1) it matches or surpasses current state-of-the-art methods in terms of sample efficiency, (2) it substantially reduces the computational cost compared to REDQ and DroQ, (3) it is easy to implement, requiring just a few lines of code on top of SAC.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Understanding Behavioral Metric Learning: A Large-Scale Study on Distracting Reinforcement Learning Environments

    cs.LG 2025-05 conditional novelty 7.0 of 10

    Across noisy DeepMind Control tasks, explicit bisimulation-metric losses add little denoising benefit beyond plain self-prediction and feature normalization, which dominate performance.

  2. Tactile MNIST: Benchmarking Active Tactile Perception

    cs.RO 2025-06 conditional novelty 6.0 of 10

    The authors release a Gymnasium-compatible benchmark with four active tactile tasks, 13,580 3D digit models, and 153,600 real touches on 600 printed digits.

  3. Hadamax Encoding: Elevating Performance in Model-Free Atari

    cs.LG 2025-05 conditional novelty 6.0 of 10

    Hadamax, a Hadamard-product and max-pooling encoder, improves PQN's median human-normalized Atari-57 score by about 80% with no algorithmic changes.

  4. Relative Value Learning

    cs.LG 2026-07 conditional novelty 5.0 of 10

    A critic that learns antisymmetric value differences ∆(s_i,s_j)=V(s_i)−V(s_j) has a provably contracting Bellman operator and an unbiased advantage estimator, and PPO with this critic matches standard PPO on Atari.

  5. Diffusion-Augmented Markov Decision Processes for Maximum Entropy Reinforcement Learning

    cs.LG 2025-12 conditional novelty 5.0 of 10

    Diffusion policies can be inserted into maximum-entropy RL by minimizing an upper bound on reverse KL, yielding DiffPPO, DiffSAC, and DiffWPO.

Pith tools