Pith. sign in

REVIEW 5 cited by

Dropout Q-Functions for Doubly Efficient Reinforcement Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2110.02034 v2 pith:6RLUIZWF submitted 2021-10-05 cs.LG cs.AI

Takuya Hiraoka , Takahisa Imagawa , Taisei Hashimoto , Takashi Onishi , Yoshimasa Tsuruoka This is my paper
classification cs.LGcs.AI
keywords efficiencyredqdropoutefficientq-functionssamplecomputationalcomputationally
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Randomized ensembled double Q-learning (REDQ) (Chen et al., 2021b) has recently achieved state-of-the-art sample efficiency on continuous-action reinforcement learning benchmarks. This superior sample efficiency is made possible by using a large Q-function ensemble. However, REDQ is much less computationally efficient than non-ensemble counterparts such as Soft Actor-Critic (SAC) (Haarnoja et al., 2018a). To make REDQ more computationally efficient, we propose a method of improving computational efficiency called DroQ, which is a variant of REDQ that uses a small ensemble of dropout Q-functions. Our dropout Q-functions are simple Q-functions equipped with dropout connection and layer normalization. Despite its simplicity of implementation, our experimental results indicate that DroQ is doubly (sample and computationally) efficient. It achieved comparable sample efficiency with REDQ, much better computational efficiency than REDQ, and comparable computational efficiency with that of SAC.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reinforcement learning entangling operations on spin qubits

    quant-ph 2025-08 conditional novelty 6.0 of 10

    An RL agent learns robust CNOT entangling pulses for realistically modeled GaAs singlet-triplet qubits and matches a gradient-based benchmark in simulation.

  2. Squeeze the Soaked Sponge: Efficient Off-policy Reinforcement Finetuning for Large Language Model

    cs.LG 2025-07 conditional novelty 6.0 of 10

    ReMix cuts reinforcement finetuning rollout volume by 30x to 450x on math reasoning by mixing historical and on-policy data with a convex KL constraint and a mid-training switch to on-policy updates.

  3. The Courage to Stop: Overcoming Sunk Cost Fallacy in Deep Reinforcement Learning

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Introduces LEAST, an adaptive early-episode-stopping rule for off-policy deep RL that improves learning efficiency on MuJoCo and DeepMind Control benchmarks.

  4. ReBRAC-v2: The Return of the King

    cs.LG 2026-08 conditional novelty 5.0 of 10

    A fixed-recipe offline RL method combining normalizing-flow actors, categorical critics, staged training, and test-time refinement beats recent flow-based baselines by 22.5 points averaged over ten OGBench categories.

  5. Deep Reinforcement Learning: From First Principles to Reasoning Models

    eess.SY 2026-07 unverdicted novelty 1.0 of 10

    A textbook survey of deep reinforcement learning, from Bellman foundations to DQN, PPO, MuZero, offline RL, and reasoning models, with UAV/SD-WAN examples throughout.

Pith tools