Robust Autonomy Emerges from Self-Play
Reviewed by Pithpith:ORJTFBLYopen to challenge →
read the original abstract
Self-play has powered breakthroughs in two-player and multi-player games. Here we show that self-play is a surprisingly effective strategy in another domain. We show that robust and naturalistic driving emerges entirely from self-play in simulation at unprecedented scale -- 1.6~billion~km of driving. This is enabled by Gigaflow, a batched simulator that can synthesize and train on 42 years of subjective driving experience per hour on a single 8-GPU node. The resulting policy achieves state-of-the-art performance on three independent autonomous driving benchmarks. The policy outperforms the prior state of the art when tested on recorded real-world scenarios, amidst human drivers, without ever seeing human data during training. The policy is realistic when assessed against human references and achieves unprecedented robustness, averaging 17.5 years of continuous driving between incidents in simulation.
This paper has not been read by Pith yet.
Forward citations
Cited by 14 Pith papers
-
Fail2Drive: Benchmarking Closed-Loop Driving Generalization
Fail2Drive is the first paired-route benchmark for closed-loop generalization in CARLA, showing an average 22.8% success-rate drop on shifted scenarios and revealing failure modes such as ignoring visible LiDAR objects.
-
Reward-Conditioned Attention: How Reward Design Shapes What Autonomous Driving Agents See
Reward design in Perceiver-based RL driving agents determines attention allocation to elements like GPS tokens and risks, inducing a learned vigilance prior and sometimes reversing attention-risk correlations, with a ...
-
Beyond Self-Play: Hierarchical Reasoning for Continuous Motion in Closed-Loop Traffic Simulation
A hierarchical Stackelberg MARL plus continuous-motion architecture with hybrid co-training produces smoother and safer closed-loop traffic behavior than standard self-play methods.
-
Goal-Oriented Reactive Simulation for Closed-Loop Trajectory Prediction
Closed-loop on-policy training with a reactive goal-oriented scene decoder cuts collision rates by up to 79.5% in dense traffic compared to standard open-loop baselines.
-
Pseudo-Expert Regularized Offline RL for End-to-End Autonomous Driving in Photorealistic Closed-Loop Environments
Pseudo-expert regularized offline RL reduces collisions and improves route completion for camera-based driving models trained on fixed simulator datasets from nuScenes.
-
SimScale: Learning to Drive via Real-World Simulation at Scale
SimScale synthesizes unseen driving states from real logs via neural rendering and reactive environments, generates pseudo-expert trajectories, and shows that co-training on real plus simulated data improves planning ...
-
ReSim: Reliable World Simulation for Autonomous Driving
ReSim is a controllable video world model trained on heterogeneous real and simulated driving data that achieves higher fidelity and controllability for both expert and non-expert actions, plus a Video2Reward module f...
-
EvoFlock: evolved inverse design of multi-agent motion
Genetic algorithm optimizes parameters of multi-agent flocking models to match user-defined objectives, with alignment emerging from spacing maintenance.
-
TerraTransfer: Learning End-to-End Driving Policies Without Expert Demonstrations
TerraTransfer decouples self-play policy pretraining from vision alignment via KL divergence and low-rank loss to produce end-to-end driving policies without expert demonstrations, matching prior methods on closed-loo...
-
Human-like autonomy emerges from self-play and a pinch of human data
Self-play RL regularized with 30 minutes of human data produces driving policies that coordinate with humans, training in 15 hours on one GPU with 2500x less data than imitation learning.
-
A High-Throughput Compute-Efficient POMDP Hide-And-Seek-Engine (HASE) for Multi-Agent Operations
A C++ Dec-POMDP simulator using data-oriented design and zero-copy PyTorch integration achieves up to 33 million steps per second on a 16-core CPU, enabling multi-agent policy training in minutes with PPO, DQN, and SAC.
-
Artificial Intelligence for Modeling and Simulation of Mixed Automated and Human Traffic
This survey synthesizes AI techniques for mixed autonomy traffic simulation and introduces a taxonomy spanning agent-level behavior models, environment-level methods, and cognitive/physics-informed approaches.
-
Toward Efficient and Robust Behavior Models for Multi-Agent Driving Simulation
An instance-centric representation with local frames, relative positional encodings, and adaptive reward transformation in adversarial IRL yields scalable, accurate, and robust behavior models for multi-agent driving ...
-
Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learning
PODS applies max-variance down-sampling to GRPO rollouts in LLM RLVR, delivering at least 1.7x faster training to peak test accuracy on reasoning benchmarks.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.