REVIEW 6 cited by
Accelerated Primal-Dual Policy Optimization for Safe Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Constrained Markov Decision Process (CMDP) is a natural framework for reinforcement learning tasks with safety constraints, where agents learn a policy that maximizes the long-term reward while satisfying the constraints on the long-term cost. A canonical approach for solving CMDPs is the primal-dual method which updates parameters in primal and dual spaces in turn. Existing methods for CMDPs only use on-policy data for dual updates, which results in sample inefficiency and slow convergence. In this paper, we propose a policy search method for CMDPs called Accelerated Primal-Dual Optimization (APDO), which incorporates an off-policy trained dual variable in the dual update procedure while updating the policy in primal space with on-policy likelihood ratio gradient. Experimental results on a simulated robot locomotion task show that APDO achieves better sample efficiency and faster convergence than state-of-the-art approaches for CMDPs.
Forward citations
Cited by 6 Pith papers
-
Scalable Constrained Multi-Agent Reinforcement Learning via State Augmentation and Consensus for Separable Dynamics
A consensus-based distributed algorithm for constrained MARL with separable dynamics achieves linear scalability and bounded constraint violations through state-augmented policies and dual variable agreement.
-
Stationary Robust Mean-Field Games under Model Mismatches
Develops infinite-horizon stationary robust mean-field games incorporating distributional uncertainty, proves equilibrium existence via fixed-point on contractive Bellman operator, gives convergent algorithm, and deri...
-
PNAct: Crafting Backdoor Attacks in Safe Reinforcement Learning
PNAct trains a safe RL agent to take unsafe actions only when a hidden trigger is present, while keeping normal safe behavior and reward when the trigger is absent.
-
Tilted Quantile Gradient Updates for Quantile-Constrained Reinforcement Learning
TQPO estimates gradients of quantile safety constraints directly through sampling and adds a tilted update to the Lagrange multiplier, improving return while satisfying the constraints.
-
Control Synthesis with Reinforcement Learning: A Modeling Perspective
A simplified linear training model yields an RL cart-pole controller that fails in physical deployment, while a high-fidelity nonlinear model yields a deployable, disturbance-robust controller.
-
Effective Reward Specification in Deep Reinforcement Learning
A thesis presenting four methods (ASAF, TeamReg, CoachReg, constrained RL, goal-conditioned GFlowNets) that improve reward specification for deep RL through demonstrations, policy regularization, behavior constraints,...
Discussion (0). Continue with ORCID to comment.