REVIEW 11 cited by
A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Deep Reinforcement Learning (RL) has been explored and verified to be effective in solving decision-making tasks in various domains, such as robotics, transportation, recommender systems, etc. It learns from the interaction with environments and updates the policy using the collected experience. However, due to the limited real-world data and unbearable consequences of taking detrimental actions, the learning of RL policy is mainly restricted within the simulators. This practice guarantees safety in learning but introduces an inevitable sim-to-real gap in terms of deployment, thus causing degraded performance and risks in execution. There are attempts to solve the sim-to-real problems from different domains with various techniques, especially in the era with emerging techniques such as large foundations or language models that have cast light on the sim-to-real. This survey paper, to the best of our knowledge, is the first taxonomy that formally frames the sim-to-real techniques from key elements of the Markov Decision Process (State, Action, Transition, and Reward). Based on the framework, we cover comprehensive literature from the classic to the most advanced methods including the sim-to-real techniques empowered by foundation models, and we also discuss the specialties that are worth attention in different domains of sim-to-real problems. Then we summarize the formal evaluation process of sim-to-real performance with accessible code or benchmarks. The challenges and opportunities are also presented to encourage future exploration of this direction. We are actively maintaining a repository to include the most up-to-date sim-to-real research work to help domain researchers.
Forward citations
Cited by 11 Pith papers
-
DADiff: Diffusion-Driven Cross-Domain Policy Adaptation for Reinforcement Learning
DADiff estimates cross-domain dynamics mismatch from diffusion-model latent-state trajectories and uses it for reward modification or data selection in policy adaptation.
-
Rationality Measurement and Theory for Reinforcement Learning Agents
The paper defines the rational risk gap for RL agents and proves it is controlled by Wasserstein distances of environment shift and Rademacher complexity of the value function class.
-
SSRL: Self-Search Reinforcement Learning
SSRL, a training pipeline that uses an LLM's own repeated sampling as a search environment for RL, improves question answering without external tools and transfers to real search engines.
-
Joint-Local Grounded Action Transformation for Sim-to-Real Transfer in Multi-Agent Traffic Control
JL-GAT extends grounded action transformation to multi-agent traffic signal control by feeding each agent's grounding models with neighboring state and action information, reducing the sim-to-real gap in simulated rai...
-
DeepShade: Enable Shade Simulation by Text-conditioned Image Generation
A text-conditioned diffusion model, trained on Blender-simulated shadows aligned to satellite images, generates building shade maps for arbitrary times and feeds them into shade-aware route planning.
-
Learning human-to-robot handovers through 3D scene reconstruction
A handover policy trained only on images rendered from a sparse-view Gaussian Splatting scene can deploy on a real robot without real-robot training data.
-
Linear Mixture Distributionally Robust Markov Decision Processes
Introduces linear mixture distributionally robust MDPs and proves offline suboptimality bounds of order 1/sqrt(K) for TV, KL, and chi-squared uncertainty sets.
-
Continual-RL for Generalization in Autonomous Racing on the RoboRacer Platform
SAC plus Continual Backpropagation, trained only on real multi-track data, fine-tunes in ~15 minutes on an unseen lower-friction RoboRacer track and outperforms MAP and MPC.
-
Foundations of Reinforcement Learning and Control:Connections and New Perspectives
A SAC-trained Half-Cheetah policy paired with a low-level model-reference adaptive controller recovers running performance after a change in joint damping, where the fixed learned policy alone fails.
-
Data Pyramid for Embodied Manipulation
Embodied training data form a five-layer pyramid—real-robot, UMI, ego/exo, simulation, general V–L—ordered by the trade-off between scale and robot alignment, and model capabilities track how those layers are mixed.
-
Foundation Model Driven Robotics: A Comprehensive Review
A review of foundation-model-driven robotics that synthesizes recent work across perception, planning, control, HRI, simulation, and sim-to-real transfer, and highlights open challenges.
Discussion (0). Sign in to comment.