Pith. sign in

REVIEW 2 major objections 5 minor 68 references

Dream to Drive with Predictive Individual World Model

T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PIWM builds a world model that tracks each surrounding vehicle individually and learns their intentions via trajectory prediction, gaining 18.81 percentage points in success rate over DreamerV3 on real-world intersection log-replay…

desk verdict A well-engineered DreamerV3 extension with credible but log-replay-bound results; the reactive-driving claim outruns the benchmark. read the letter →

arxiv 2501.16733 v1 pith:VKIVU7OD submitted 2025-01-28 cs.RO cs.CVcs.LG

Yinfeng Gao , Qichao Zhang , Da-wei Ding , Dongbin Zhao This is my paper · ORCID
classification cs.ROcs.CVcs.LG
keywords predictiveindividualworldmodelmodel-basedreinforcementlearningautonomousdrivinginteractivetrajectorypredictionmodelslog-replaysimulationINTERACTIONdatasetlatentintentionmodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a model-based reinforcement learning agent drives better in dense, interactive intersections when its world model represents each vehicle separately rather than compressing the whole scene into one latent state. The method, PIWM, splits vehicles into ego, direct-influence, and potential-influence branches, relates them with self-attention, and trains the latent states to predict future trajectories instead of reconstructing observations. On intersection scenarios replayed from the INTERACTION dataset, PIWM reaches 85.00% success in eight highly interactive maps and 74.79% on 156 unseen scenarios, beating DreamerV3 by 18.81 percentage points in the large-scale benchmark at matched model size. If correct, this points to a concrete way to make world models intention-aware, which is a key obstacle to reactive urban driving.

What carries the argument

The predictive individual world model (PIWM). It replaces one scene-level recurrent state-space model with branched encoders and gated recurrent units for the ego, direct-influence vehicles (VDI), and potential-influence vehicles (VPI). A self-attention layer over the deterministic states produces vehicle-to-vehicle interaction features, and the decoder predicts the next two seconds of trajectories for the ego and VDI rather than reconstructing observations. This trajectory prediction is the representation-learning engine: it forces latent states to encode long-term motion trends, and the actor-critic behavior model, trained in imagination over these intention-aware states, outputs a discrete desired longitudinal speed.

What would settle it

Train and evaluate PIWM in the same eight and 156 intersection scenarios but with background vehicles that respond to the ego, such as braking, yielding, or accelerating when the ego approaches. If PIWM's success-rate margin over DreamerV3 collapses or its collision rate rises sharply in such a reactive simulator, the claim that trajectory-prediction states capture road users' intentions would be falsified.

Watch

Extended reading notes

Core claim

The central discovery is that replacing scene-level reconstruction with individual-level, prediction-driven representation learning in a world model substantially improves closed-loop driving performance. PIWM combines branched per-vehicle latent states, self-attention interaction modeling, and a trajectory prediction objective inside a DreamerV3-style imagination framework. On the paper's benchmarks it achieves the best safety and efficiency among random, DQN, PPO, SAC, and DreamerV3, with success rate 85.00% versus 40.00% on the small-scale set and 74.79% versus 55.98% on the large-scale set, plus lower collision rates at every traffic density. The decoded predicted trajectories also show that the latent states carry something like vehicle intentions, which is the mechanism the authors argue explains the gain.

Load-bearing premise

The load-bearing assumption is that the evaluation environment's background vehicles replay fixed recorded tracks and do not react to the ego car, so the claimed interactive, intention-aware driving is only tested against non-responsive traffic.

Editorial extensions

If this is right

  • Individual-level world modeling plus trajectory-prediction representation learning is enough to raise success rate by 18.81 percentage points over a scene-level world model at the same model size.
  • The policy trained in imagination transfers to unseen intersection maps, not just the training scenarios, with 74.79% success and the fewest collisions in low-, medium-, and high-density traffic.
  • Predicted future trajectories can be decoded from latent states during deployment, giving a human-viewable explanation of why the agent acts.
  • Ablations attribute the gain more to interaction prediction than to branching alone: adding trajectory prediction to individual modeling raises success from 50.00% to 85.00% in the small-scale experiment and from 61.11% to 74.79% in the large-scale experiment.
  • Because the evaluation only used non-reactive background traffic, the reported numbers measure performance against replayed tracks, not against drivers who respond to the ego vehicle.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the benefit comes from intention-aware states, PIWM's advantage should be largest where interaction is frequent, such as high-density unprotected turns, and smallest in sparse traffic; the paper's density analysis is consistent with this but does not directly test it as a prediction.
  • Editorial inference: the individual-branch plus prediction-as-representation recipe may transfer to other multi-agent control problems, such as robot navigation among pedestrians, where a target's future path is a natural supervised signal.
  • Editorial inference: because the action space is discrete longitudinal speed on a fixed route, an important stress test is full lateral control; the gains could shrink when the policy must also choose headings and merge points.
  • Editorial inference: a reactive-traffic version of the same benchmarks, where background vehicles brake, yield, or cut in based on the ego's actions, would directly test whether the learned 'intentions' are genuine interaction models or pattern matching on recorded clips.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes PIWM, a model-based reinforcement learning method for autonomous driving in urban intersections. PIWM models each vehicle with a branched recurrent state-space model, uses self-attention over vehicle latent states to capture interactions, and replaces observation reconstruction with a trajectory-prediction objective (Eq. 7) to obtain 'intention-aware' states. A DreamerV3-style actor-critic is trained in imagination. Experiments on INTERACTION-derived log-replay scenarios (I-SIM) show PIWM outperforms DreamerV3 and model-free baselines in success rate, collision rate, and completion ratio on both small-scale and large-scale benchmarks.

Significance. The paper makes a useful engineering contribution: PIWM replaces scene-level world-model reconstruction with an individual-level, branched latent representation and an auxiliary trajectory-prediction task, and it demonstrates substantial empirical gains over DreamerV3 and model-free baselines in log-replay urban intersection benchmarks. The comparison is reasonably controlled (comparable parameter counts, same network structures, three runs with standard deviations, ablations over the individual-modeling and prediction components). The results are reproducible in principle: code and pretrained models are provided. However, the motivating claim of reactive driving is not tested by the chosen benchmark, so the significance is contingent on rescoping the claims or adding reactive experiments.

major comments (2)
  1. [IV-A2, V, Eq. (7)] The benchmark cannot validate the paper's central claim of reactive driving and intention inference. Section IV-A2 states that I-SIM is a log-replay simulator in which 'the behaviors of other road users are replayed from the log record,' and Section V concedes that 'agents are trained and evaluated with non-reactive background traffic.' Under this protocol, the future trajectories y_t used in the prediction log-loss of Eq. (7) are fixed once the episode is drawn from the dataset and are independent of the ego's actions; for background vehicles they are literally the recorded tracks. The latent states trained by Eq. (7) may therefore encode correlations in the logged data rather than infer contingent intentions of agents that respond to the ego. Consequently, the headline results (Table I: 85.00% vs 40.00% small-scale, 74.79% vs 55.98% large-scale) establish strong performance on non-reactive log replay but do not support the abstract's and Section I's claims of 'reactive driving behaviors' and learning 'vehicles' long-term intentions.' Because the limitation is disclosed, the fix is feasible: either add experiments in a simulator with reactive background agents, or explicitly rescope the abstract, introduction, and conclusion to non-reactive log-replay evaluation.
  2. [IV-D2, Fig. 6] The claim of 'sound interactive ability' based on collision counts across traffic-density levels is not supported by the experimental setup. Since background vehicles do not react to the ego, the number of collisions in a scenario is determined almost entirely by the ego's speed control against fixed tracks; the random-policy baseline already achieves 41.67% success in the large-scale test set (Table I), indicating that a substantial fraction of scenarios are trivially passable. The density-stratified collision analysis is informative for non-reactive settings but cannot be read as evidence of interactive competence. The text should avoid attributing the gains to 'interactive reasoning' unless reactive evaluation is added.
minor comments (5)
  1. [IV-A2, ref. [37]] Reference [37] is cited as the source of I-SIM, but the listed title is 'Trajgen: Generating realistic and diverse trajectories with reactive and feasible agent behaviors'; please clarify whether I-SIM is the log-replay component of Trajgen or a separate simulator, and correct the reference if needed.
  2. [Eq. (7)] The KL-divergence terms are written as βKL[q(z_t | h_t, x_t) || p(bar z_t | h_t)], which omits the self-attention variable self_att_t from Eq. (3); this shorthand is confusing and should be aligned with the notation in Eqs. (2)-(3).
  3. [Fig. 5] The training curves in Fig. 5 do not show the variability across the three runs mentioned in Section IV-D; please add error bars or shaded regions.
  4. [IV-A3, IV-E2] Section IV-A3 says the prediction horizon is set to 2s in the following experiments, while the ablation study in Section IV-E2 uses H+ in units of steps; state explicitly that H+ = 20 corresponds to 2s at the 10 Hz control frequency.
  5. [Table I] Consider explicitly defining the relationship 'Failure Rate = Collision + Time-exceed' in the caption, since the current multi-row header is hard to parse.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: PIWM is an empirical RL system, the trajectory-prediction objective supervises against logged futures rather than the success metric, and the only notable self-citation (I-SIM, ref. [37]) is an open-source simulator built on the external INTERACTION dataset.

full rationale

The paper does not contain a derivation that reduces to its own inputs. In Eq. (7), the world-model loss combines prediction log-likelihoods for logged future trajectories, reward and continuation log-likelihoods, and KL terms; the actor-critic in Eqs. (9) and (11) is trained on imagined rollouts of that world model. None of these terms is the reported success rate, so the headline results (85.00% vs 40.00% and 74.79% vs 55.98% in Table I) are not fitted by construction. The trajectory-prediction ground truth y_t is "derived from the data" (Section III-D) and stored in the replay buffer via Algorithm 1, but it is environment data, not the model's own output, so calling the decoder a prediction module is not circular. The main self-citation cluster is I-SIM (ref. [37], overlapping authorship): "we use the I-SIM[37], which is an open-sourced log-replay simulator for the INTERACTION dataset" (Section IV-A2). This is a load-bearing experimental tool in the sense that all results are generated with it, but it is published, open-source, and built on the external INTERACTION dataset, so it does not smuggle in the paper's conclusion. The conclusion's limitation statement, "agents are trained and evaluated with non-reactive background traffic" (Section V), is a genuine external-validity caveat: because social vehicles replay recorded tracks, the Eq. (7) predictions are for a fixed logged future and the benchmark cannot test contingent reactions by other agents. That concern bears on correctness and generalization, not on circularity, because the policy is not fitted to the success metric and the auxiliary prediction targets are independent quantities. Overall, the central empirical claim stands on its own comparison against DreamerV3 and model-free baselines; the self-citations are contextual or instrumental, not definitional.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests mainly on simulation-environment choices (non-reactive log-replay) and on hand-designed reward and split parameters, plus the assumption that trajectory prediction encodes intentions. No new physical entities are introduced, so invented_entities is empty.

free parameters (6)
  • reward weights (rcollision = -30*(1+v_norm), rspeed = 0.3*v_norm, rstep = -0.3) = -30 collision scale, 0.3 speed gain, -0.3 step penalty
    Hand-chosen trade-off between safety and efficiency; directly shapes the policy and the reported safety/efficiency metrics.
  • VDI/VPI split (Nd=5, Np=5, detection range 30m behind and 60m otherwise) = 5 VDI, 5 VPI, N=10
    Authors state the split 'only has a minor impact' but provide no systematic tuning; it determines which vehicles get trajectory decoders and attention in the world model.
  • prediction horizon = 20 steps (2s), selected by ablation
    Chosen from Fig. 8 ablation to balance success rate and average distance error; affects what latent states encode.
  • dataset inclusion thresholds = length<=5.5m, duration>=5s, track>=20m
    Filters 732 recorded scenarios to 640 'meaningful' ones for training and evaluation; post-hoc selection that defines the benchmark.
  • action set = {0, 3, 6, 9} m/s
    Hand-designed discrete speed choices that limit the policy's expressiveness and the scope to longitudinal control.
  • number of surrounding vehicles = N=10
    Observation truncation, zero-padded when fewer vehicles; assumption that more distant vehicles are irrelevant.
assumptions (5)
  • domain assumption Log-replay with non-reactive background traffic is an adequate proxy for interactive urban driving.
    Section IV-A.2 and the conclusion state 'agents are trained and evaluated with non-reactive background traffic'; the central 'reactive' claim depends on this.
  • domain assumption Vehicle detection and localization are already solved, so vector poses are the observation.
    Section I: 'we first assume that vehicle detection and localization are realized by existing perception algorithms'.
  • ad hoc to paper Trajectory prediction loss encodes vehicles' long-term intentions in latent states.
    Core mechanism of PIWM (Section III-B, Eq. 7); no direct measurement of intention quality, only indirect success-rate gains.
  • domain assumption VDI and VPI membership is fixed during imagination.
    Section III-D: 'We keep the members of VDI and VPI the same as in the first state during imagination'; real interactions would change these sets.
  • domain assumption Reward as defined (collision penalty scaled by speed, speed reward, step penalty) captures safety and efficiency.
    Section IV-A.5; comfort and courtesy are explicitly omitted, so the reward only approximates the stated objectives.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dream to Drive with Predictive Individual World Model." pith.science (2026). https://pith.science/paper/VKIVU7OD

@misc{pith2026250116733,
  author       = {Pith},
  title        = {Pith review of: Dream to Drive with Predictive Individual World Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VKIVU7OD}},
  note         = {Machine review of arXiv:2501.16733}
}
read the original abstract

It is still a challenging topic to make reactive driving behaviors in complex urban environments as road users' intentions are unknown. Model-based reinforcement learning (MBRL) offers great potential to learn a reactive policy by constructing a world model that can provide informative states and imagination training. However, a critical limitation in relevant research lies in the scene-level reconstruction representation learning, which may overlook key interactive vehicles and hardly model the interactive features among vehicles and their long-term intentions. Therefore, this paper presents a novel MBRL method with a predictive individual world model (PIWM) for autonomous driving. PIWM describes the driving environment from an individual-level perspective and captures vehicles' interactive relations and their intentions via trajectory prediction task. Meanwhile, a behavior policy is learned jointly with PIWM. It is trained in PIWM's imagination and effectively navigates in the urban driving scenes leveraging intention-aware latent states. The proposed method is trained and evaluated on simulation environments built upon real-world challenging interactive scenarios. Compared with popular model-free and state-of-the-art model-based reinforcement learning methods, experimental results show that the proposed method achieves the best performance in terms of safety and efficiency.

Figures

Figures reproduced from arXiv: 2501.16733 by the authors.

Figure 1
Figure 1. Comparison of Dreamer[25] and our method. Considering a complex driving scenario with an ego vehicle and several social vehicles. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The detailed structure of PIWM and Behavior Model. The modules represented by solid lines are branched-only, the other modules [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Typical scenarios of the experiments. Where the ego vehicle [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The workflow of the I-SIM simulator. The simulation’s [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Training curves of our proposed method PIWM and other learning-based baselines. PIWM shows superior performance in terms of [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Collision numbers of different methods in the evaluation phase [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Ablation study of the designs of the PIWM. Individual modeling helps to improve sample efficiency and final performance significantly [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Ablation study of prediction horizons on both benchmark [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Simulated scenarios with sampled key timestamps. Two typical driving scenarios, cut-in and unprotected left turn, are selected, [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 54 canonical work pages

  1. [1]

    Interactive trajectory prediction using a driving risk map-integrated deep learning method for surrounding vehicles on highways,

    X. Liu, Y . Wang, K. Jiang, Z. Zhou, K. Nam, and C. Yin, “Interactive trajectory prediction using a driving risk map-integrated deep learning method for surrounding vehicles on highways,” IEEE Transactions on Intelligent Transportation Systems , vol. 23, no. 10, pp. 19 076– 19 087, 2022

    work page 2022

  2. [2]

    A lidar-openstreetmap matching method for vehicle global position initialization based on boundary directional feature extraction,

    Z. Li, Y . Wang, R. Zhang, F. Ding, C. Wei, and J.- G. Lu, “A lidar-openstreetmap matching method for vehicle global position initialization based on boundary directional feature extraction,” IEEE Transactions on Intelligent Vehicles, pp. 1–13, 2024

    work page 2024

  3. [3]

    Security-based resilient triggered output feedback lane keeping control for human–machine cooperative steering intelligent heavy truck under denial-of-service attacks,

    F. Ding, H. Shan, X. Han, C. Jiang, C. Peng, and J. Liu, “Security-based resilient triggered output feedback lane keeping control for human–machine cooperative steering intelligent heavy truck under denial-of-service attacks,” IEEE TRANSACTIONS ON INTELLIGENT VEHICLES, VOL. , NO. , 2024 13 Fig. 9: Simulated scenarios with sampled key timestamps. Two typic...

    work page 2024

  4. [4]

    Human-like decision making and motion control for smooth and natural car following,

    C. Wei, E. Paschalidis, N. Merat, A. S. Crusat, F. Ha- jiseyedjavadi, and R. Romano, “Human-like decision making and motion control for smooth and natural car following,” IEEE Transactions on Intelligent Vehicles , vol. 8, no. 1, pp. 263–274, 2023

    work page 2023

  5. [5]

    Human-like control for automated vehicles and avoiding “vehicle face-off

    J. Chen, D. Sun, and M. Zhao, “Human-like control for automated vehicles and avoiding “vehicle face-off” in unprotected left turn scenarios,” IEEE Transactions on Intelligent Transportation Systems , vol. 24, no. 2, pp. 1609–1618, 2023

    work page 2023

  6. [6]

    A reasoning framework for autonomous urban driving,

    D. Ferguson, C. Baker, M. Likhachev, and J. Dolan, “A reasoning framework for autonomous urban driving,” in 2008 IEEE Intelligent Vehicles Symposium. IEEE, 2008, pp. 775–780

    work page 2008

  7. [7]

    A survey of motion planning and control techniques for self-driving urban vehicles,

    B. Paden, M. ˇC´ap, S. Z. Yong, D. Yershov, and E. Fraz- zoli, “A survey of motion planning and control techniques for self-driving urban vehicles,” IEEE Transactions on Intelligent Vehicles, vol. 1, no. 1, pp. 33–55, 2016

    work page 2016

  8. [8]

    A perception-driven autonomous urban vehicle,

    J. Leonard, J. How, S. Teller, M. Berger, S. Campbell, G. Fiore, L. Fletcher, E. Frazzoli, A. Huang, S. Karaman, et al., “A perception-driven autonomous urban vehicle,” Journal of Field Robotics , vol. 25, no. 10, pp. 727–774, 2008

    work page 2008

Show all 68 references
  1. [9]

    Hierarchical model-based imitation learning for planning in autonomous driving,

    E. Bronstein, M. Palatucci, D. Notz, B. White, A. Kue- fler, Y . Lu, S. Paul, P. Nikdel, P. Mougin, H. Chen, et al. , “Hierarchical model-based imitation learning for planning in autonomous driving,” in 2022 IEEE/RSJ In- ternational Conference on Intelligent Robots and Systems...

    2022

  2. [10]

    Waymax: An accelerated, data-driven simulator for large-scale au- tonomous driving research,

    C. Gulino, J. Fu, W. Luo, G. Tucker, E. Bronstein, Y . Lu, J. Harb, X. Pan, Y . Wang, X. Chen, et al. , “Waymax: An accelerated, data-driven simulator for large-scale au- tonomous driving research,” inThirty-seventh Conference on Neural Information Processing Systems Datasets ...

    2023

  3. [11]

    Learning to drive by imi- tation: An overview of deep behavior cloning methods,

    A. O. Ly and M. Akhloufi, “Learning to drive by imi- tation: An overview of deep behavior cloning methods,” IEEE Transactions on Intelligent Vehicles , vol. 6, no. 2, pp. 195–209, 2020

    2020

  4. [12]

    Urban driving with conditional imitation learning,

    J. Hawke, R. Shen, C. Gurau, S. Sharma, D. Reda, N. Nikolov, P. Mazur, S. Micklethwaite, N. Griffiths, A. Shah, et al., “Urban driving with conditional imitation learning,” in 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 251– 257

    2020

  5. [13]

    Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst,

    M. Bansal, A. Krizhevsky, and A. Ogale, “Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst,” in Proc. Robotics: Science and Systems (RSS), 2019

    2019

  6. [14]

    S. E. Li, Reinforcement learning for sequential decision and optimal control . Springer, 2023

    2023

  7. [15]

    Multi-task safe reinforcement learning for navigating intersections in dense traffic,

    Y . Liu, Y . Gao, Q. Zhang, D. Ding, and D. Zhao, “Multi-task safe reinforcement learning for navigating intersections in dense traffic,” Journal of the Franklin Institute, 2022

    2022

  8. [16]

    Deep reinforcement learning based game-theoretic decision-making for au- tonomous vehicles,

    M. Yuan, J. Shan, and K. Mi, “Deep reinforcement learning based game-theoretic decision-making for au- tonomous vehicles,” IEEE Robotics and Automation Let- ters, vol. 7, no. 2, pp. 818–825, 2021

    2021

  9. [17]

    Highway lane change decision-making via attention-based deep rein- forcement learning,

    J. Wang, Q. Zhang, and D. Zhao, “Highway lane change decision-making via attention-based deep rein- forcement learning,” IEEE/CAA Journal of Automatica Sinica, vol. 9, no. 3, pp. 567–569, 2021

    2021

  10. [18]

    Interaction-aware decision-making for automated vehi- cles using social value orientation,

    L. Crosato, H. P. H. Shum, E. S. L. Ho, and C. Wei, “Interaction-aware decision-making for automated vehi- cles using social value orientation,” IEEE Transactions on Intelligent Vehicles , vol. 8, no. 2, pp. 1339–1349, 2023

    2023

  11. [19]

    Deep reinforcement learning-based automatic exploration for navigation in unknown environment,

    H. Li, Q. Zhang, and D. Zhao, “Deep reinforcement learning-based automatic exploration for navigation in unknown environment,” IEEE transactions on neural networks and learning systems , vol. 31, no. 6, pp. 2064– 2076, 2019

    2019

  12. [20]

    Recurrent world models fa- cilitate policy evolution,

    D. Ha and J. Schmidhuber, “Recurrent world models fa- cilitate policy evolution,” Advances in neural information processing systems, vol. 31, 2018

    2018

  13. [21]

    Model-based reinforcement learning for atari,

    L. Kaiser, M. Babaeizadeh, P. Milos, B. Osinski, R. H. Campbell, K. Czechowski, D. Erhan, C. Finn, P. Koza- kowski, S. Levine, et al. , “Model-based reinforcement learning for atari,” arXiv preprint arXiv:1903.00374 , 2019

    1903 arXiv

  14. [22]

    Learning latent dynamics for planning from pixels,

    D. Hafner, T. Lillicrap, I. Fischer, R. Villegas, D. Ha, H. Lee, and J. Davidson, “Learning latent dynamics for planning from pixels,” in International conference on machine learning. PMLR, 2019, pp. 2555–2565

    2019

  15. [23]

    Dream to control: Learning behaviors by latent imagination,

    D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi, “Dream to control: Learning behaviors by latent imagination,” in International Conference on Learning Representations , 2019

    2019

  16. [24]

    Mas- tering atari with discrete world models,

    D. Hafner, T. P. Lillicrap, M. Norouzi, and J. Ba, “Mas- tering atari with discrete world models,” in International Conference on Learning Representations , 2020

    2020

  17. [25]

    Mas- tering diverse domains through world models,

    D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap, “Mas- tering diverse domains through world models,” arXiv preprint arXiv:2301.04104, 2023

    2023 arXiv

  18. [26]

    Steadily learn to drive with virtual memory,

    Y . Zhang, Y . Mu, Y . Yang, Y . Guan, S. E. Li, Q. Sun, and J. Chen, “Steadily learn to drive with virtual memory,” arXiv preprint arXiv:2102.08072 , 2021

    2021 arXiv

  19. [27]

    Enhance sample efficiency and robustness of end-to-end urban autonomous driving via semantic masked world model,

    Z. Gao, Y . Mu, C. Chen, J. Duan, P. Luo, Y . Lu, and S. E. Li, “Enhance sample efficiency and robustness of end-to-end urban autonomous driving via semantic masked world model,” IEEE Transactions on Intelligent Transportation Systems, pp. 1–13, 2024

    2024

  20. [28]

    Latent imagination facilitates zero-shot transfer in autonomous racing,

    A. Brunnbauer, L. Berducci, A. Brandst ´atter, M. Lechner, R. Hasani, D. Rus, and R. Grosu, “Latent imagination facilitates zero-shot transfer in autonomous racing,” in 2022 International Conference on Robotics and Automa- tion (ICRA). IEEE, 2022, pp. 7513–7520

    2022

  21. [29]

    Iso-dream: Isolating and leveraging noncontrollable visual dynam- ics in world models,

    M. Pan, X. Zhu, Y . Wang, and X. Yang, “Iso-dream: Isolating and leveraging noncontrollable visual dynam- ics in world models,” Advances in Neural Information Processing Systems, vol. 35, pp. 23 178–23 191, 2022

    2022

  22. [30]

    Deep reinforcement learning framework for autonomous IEEE TRANSACTIONS ON INTELLIGENT VEHICLES, VOL. , NO. , 2024 15 driving,

    A. E. Sallab, M. Abdou, E. Perot, and S. Yogamani, “Deep reinforcement learning framework for autonomous IEEE TRANSACTIONS ON INTELLIGENT VEHICLES, VOL. , NO. , 2024 15 driving,” arXiv preprint arXiv:1704.02532 , 2017

    2024 arXiv

  23. [31]

    Human-level control through deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Ve- ness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. , “Human-level control through deep reinforcement learning,” nature, vol. 518, no. 7540, pp. 529–533, 2015

    2015

  24. [32]

    Deep merging: Vehicle merging controller based on deep reinforcement learning with embedding network,

    I. Nishitani, H. Yang, R. Guo, S. Keshavamurthy, and K. Oguchi, “Deep merging: Vehicle merging controller based on deep reinforcement learning with embedding network,” in 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 216– 221

    2020

  25. [33]

    Outracing champion gran tur- ismo drivers with deep reinforcement learning,

    P. R. Wurman, S. Barrett, K. Kawamoto, J. MacGlashan, K. Subramanian, T. J. Walsh, R. Capobianco, A. Devlic, F. Eckert, F. Fuchs, et al., “Outracing champion gran tur- ismo drivers with deep reinforcement learning,” Nature, vol. 602, no. 7896, pp. 223–228, 2022

    2022

  26. [34]

    Soft actor-critic: Off-policy maximum entropy deep reinforce- ment learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforce- ment learning with a stochastic actor,” in International conference on machine learning . PMLR, 2018, pp. 1861–1870

    2018

  27. [35]

    Mastering arterial traffic signal control with multi-agent attention-based soft actor-critic model,

    F. Mao, Z. Li, Y . Lin, and L. Li, “Mastering arterial traffic signal control with multi-agent attention-based soft actor-critic model,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 3, pp. 3129–3144, 2023

    2023

  28. [36]

    A bi-level network-wide cooperative driving approach including deep reinforcement learning-based routing,

    J. Zhang, J. Ge, S. Li, S. Li, and L. Li, “A bi-level network-wide cooperative driving approach including deep reinforcement learning-based routing,” IEEE Trans- actions on Intelligent Vehicles , pp. 1–17, 2023

    2023

  29. [37]

    Trajgen: Generating realistic and diverse trajectories with reactive and feasible agent behaviors for autonomous driving,

    Q. Zhang, Y . Gao, Y . Zhang, Y . Guo, D. Ding, Y . Wang, P. Sun, and D. Zhao, “Trajgen: Generating realistic and diverse trajectories with reactive and feasible agent behaviors for autonomous driving,” IEEE Transactions on Intelligent Transportation Systems , vol. 23, no. 12,...

    2022

  30. [38]

    Efficient deep rein- forcement learning with imitative expert priors for au- tonomous driving,

    Z. Huang, J. Wu, and C. Lv, “Efficient deep rein- forcement learning with imitative expert priors for au- tonomous driving,” IEEE Transactions on Neural Net- works and Learning Systems , 2022

    2022

  31. [39]

    Interpretable end- to-end urban autonomous driving with latent deep re- inforcement learning,

    J. Chen, S. E. Li, and M. Tomizuka, “Interpretable end- to-end urban autonomous driving with latent deep re- inforcement learning,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 6, pp. 5068–5078, 2021

    2021

  32. [40]

    Increasing the efficiency of policy learning for autonomous vehicles by multi-task representation learning,

    E. Kargar and V . Kyrki, “Increasing the efficiency of policy learning for autonomous vehicles by multi-task representation learning,” IEEE Transactions on Intelli- gent Vehicles, vol. 7, no. 3, pp. 701–710, 2022

    2022

  33. [41]

    A survey on model-based reinforcement learning,

    F.-M. Luo, T. Xu, H. Lai, X.-H. Chen, W. Zhang, and Y . Yu, “A survey on model-based reinforcement learning,” Science China Information Sciences , vol. 67, no. 2, p. 121101, 2024

    2024

  34. [42]

    Uncertainty-aware model- based reinforcement learning: Methodology and appli- cation in autonomous driving,

    J. Wu, Z. Huang, and C. Lv, “Uncertainty-aware model- based reinforcement learning: Methodology and appli- cation in autonomous driving,” IEEE Transactions on Intelligent Vehicles, vol. 8, no. 1, pp. 194–203, 2022

    2022

  35. [43]

    Dynamic-horizon model-based value estimation with latent imagination,

    J. Wang, Q. Zhang, and D. Zhao, “Dynamic-horizon model-based value estimation with latent imagination,” IEEE Transactions on Neural Networks and Learning Systems, 2022

    2022

  36. [44]

    Model- based imitation learning for urban driving,

    A. Hu, G. Corrado, N. Griffiths, Z. Murez, C. Gurau, H. Yeo, A. Kendall, R. Cipolla, and J. Shotton, “Model- based imitation learning for urban driving,” Advances in Neural Information Processing Systems , vol. 35, pp. 20 703–20 716, 2022

    2022

  37. [45]

    Drive- dreamer: Towards real-world-driven world models for autonomous driving,

    X. Wang, Z. Zhu, G. Huang, X. Chen, and J. Lu, “Drive- dreamer: Towards real-world-driven world models for autonomous driving,” arXiv preprint arXiv:2309.09777 , 2023

    2023 arXiv

  38. [46]

    M2i: From factored marginal trajectory prediction to interactive prediction,

    Q. Sun, X. Huang, J. Gu, B. C. Williams, and H. Zhao, “M2i: From factored marginal trajectory prediction to interactive prediction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 6543–6552

    2022

  39. [47]

    Densetnt: End-to-end tra- jectory prediction from dense goal sets,

    J. Gu, C. Sun, and H. Zhao, “Densetnt: End-to-end tra- jectory prediction from dense goal sets,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 15 303–15 312

    2021

  40. [48]

    Planning-inspired hierarchical trajectory prediction via lateral-longitudinal decompo- sition for autonomous driving,

    D. Li, Q. Zhang, Z. Xia, Y . Zheng, K. Zhang, M. Yi, W. Jin, and D. Zhao, “Planning-inspired hierarchical trajectory prediction via lateral-longitudinal decompo- sition for autonomous driving,” IEEE Transactions on Intelligent Vehicles, pp. 1–12, 2023

    2023

  41. [49]

    Social interaction-aware dynamical models and decision-making for autonomous vehicles,

    L. Crosato, K. Tian, H. P. Shum, E. S. Ho, Y . Wang, and C. Wei, “Social interaction-aware dynamical models and decision-making for autonomous vehicles,” Advanced Intelligent Systems , 2023, due to be Gold (journal is fully OA). [Online]. Available: https://durham-repository.w...

    2023

  42. [50]

    Rethinking integration of prediction and plan- ning in deep learning-based automated driving systems: A review,

    S. Hagedorn, M. Hallgarten, M. Stoll, and A. Con- durache, “Rethinking integration of prediction and plan- ning in deep learning-based automated driving systems: A review,” arXiv preprint arXiv:2308.05731 , 2023

    2023 arXiv

  43. [51]

    Differentiable integrated motion prediction and planning with learnable cost function for autonomous driving,

    Z. Huang, H. Liu, J. Wu, and C. Lv, “Differentiable integrated motion prediction and planning with learnable cost function for autonomous driving,”IEEE transactions on neural networks and learning systems , 2023

    2023

  44. [52]

    Solution concepts in hierar- chical games under bounded rationality with applications to autonomous driving,

    A. Sarkar and K. Czarnecki, “Solution concepts in hierar- chical games under bounded rationality with applications to autonomous driving,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 6, 2021, pp. 5698–5708

    2021

  45. [53]

    Generalized dynamic cognitive hierarchy models for strategic driving behavior,

    A. Sarkar, K. Larson, and K. Czarnecki, “Generalized dynamic cognitive hierarchy models for strategic driving behavior,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 36, no. 5, 2022, pp. 5173– 5182

    2022

  46. [54]

    Gameformer: Game- theoretic modeling and learning of transformer-based interactive prediction and planning for autonomous driv- ing,

    Z. Huang, H. Liu, and C. Lv, “Gameformer: Game- theoretic modeling and learning of transformer-based interactive prediction and planning for autonomous driv- ing,” in Proceedings of the International Conference on Computer Vision, 2023

    2023

  47. [55]

    Gpt- driver: Learning to drive with gpt,

    J. Mao, Y . Qian, J. Ye, H. Zhao, and Y . Wang, “Gpt- driver: Learning to drive with gpt,” in NeurIPS 2023 IEEE TRANSACTIONS ON INTELLIGENT VEHICLES, VOL. , NO. , 2024 16 Foundation Models for Decision Making Workshop , 2023

    2023

  48. [56]

    Dilu: A knowledge-driven approach to autonomous driving with large language models,

    L. Wen, D. Fu, X. Li, X. Cai, M. Tao, P. Cai, M. Dou, B. Shi, L. He, and Y . Qiao, “Dilu: A knowledge-driven approach to autonomous driving with large language models,” in The Twelfth International Conference on Learning Representations, 2023

    2023

  49. [57]

    Lmdrive: Closed-loop end-to-end driv- ing with large language models,

    H. Shao, Y . Hu, L. Wang, S. L. Waslander, Y . Liu, and H. Li, “Lmdrive: Closed-loop end-to-end driv- ing with large language models,” arXiv preprint arXiv:2312.07488, 2023

    2023 arXiv

  50. [58]

    Social attention for au- tonomous decision-making in dense traffic,

    E. Leurent and J. Mercat, “Social attention for au- tonomous decision-making in dense traffic,” in Machine Learning for Autonomous Driving Workshop at NeurIPS 2019, 2019

    2019

  51. [59]

    A dis- tributional perspective on reinforcement learning,

    M. G. Bellemare, W. Dabney, and R. Munos, “A dis- tributional perspective on reinforcement learning,” in International conference on machine learning . PMLR, 2017, pp. 449–458

    2017

  52. [60]

    R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press, 2018

    2018

  53. [61]

    Simple statistical gradient-following al- gorithms for connectionist reinforcement learning,

    R. J. Williams, “Simple statistical gradient-following al- gorithms for connectionist reinforcement learning,” Ma- chine learning, vol. 8, pp. 229–256, 1992

    1992

  54. [62]

    Mastering atari, go, chess and shogi by planning with a learned model,

    J. Schrittwieser, I. Antonoglou, T. Hubert, K. Simonyan, L. Sifre, S. Schmitt, A. Guez, E. Lockhart, D. Hassabis, T. Graepel, et al., “Mastering atari, go, chess and shogi by planning with a learned model,” Nature, vol. 588, no. 7839, pp. 604–609, 2020

    2020

  55. [63]

    Interaction dataset: An inter- national, adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps,

    W. Zhan, L. Sun, D. Wang, H. Shi, A. Clausse, M. Naumann, J. Kummerle, H. Konigshof, C. Stiller, A. de La Fortelle, et al. , “Interaction dataset: An inter- national, adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps,” arXiv preprin...

    1910 arXiv

  56. [64]

    Contingencies from observations: Tractable contingency planning with learned behavior models,

    N. Rhinehart, J. He, C. Packer, M. A. Wright, R. McAl- lister, J. E. Gonzalez, and S. Levine, “Contingencies from observations: Tractable contingency planning with learned behavior models,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, p...

    2021

  57. [65]

    Vectornet: Encoding hd maps and agent dy- namics from vectorized representation,

    J. Gao, C. Sun, H. Zhao, Y . Shen, D. Anguelov, C. Li, and C. Schmid, “Vectornet: Encoding hd maps and agent dy- namics from vectorized representation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 525–11 533

    2020

  58. [66]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347 , 2017

    2017 arXiv

  59. [67]

    Soft actor-critic for discrete action settings,

    P. Christodoulou, “Soft actor-critic for discrete action settings,” arXiv preprint arXiv:1910.07207 , 2019

    1910 arXiv

  60. [68]

    Challenges and opportunities in offline reinforcement learning from visual observations,

    C. Lu, P. J. Ball, T. G. J. Rudner, J. Parker- Holder, M. A. Osborne, and Y . W. Teh, “Challenges and opportunities in offline reinforcement learning from visual observations,” Transactions on Machine Learning Research , 2023. [Online]. Available: https: //openreview.net/forum...

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.