Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

DiffE2E: Rethinking End-to-End Driving with a Hybrid Action Diffusion and Supervised Policy

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DiffE2E's central claim: end-to-end driving is best served by a hybrid policy in which a diffusion model generates the trajectory distribution and a supervised head predicts key control variables in the same Transformer decoder.

desk verdict Genuine architectural idea and strong ablations, but the CARLA SOTA claim is not verifiable until the custom Longest6 protocol is fully specified. read the letter →

arxiv 2505.19516 v1 pith:RG4CVBGC submitted 2025-05-26 cs.RO

Rui Zhao , Yuze Fan , Ziguo Chen , Fei Gao , Zhenhai Gao This is my paper
classification cs.RO
keywords end-to-endautonomousdrivingdiffusionpolicyhybriddiffusion-supervisiondecodermultimodalsensorfusiontrajectoryplanningCARLAclosed-loopbenchmarkNAVSIM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that the best end-to-end driving policy is neither pure imitation nor pure diffusion: one branch of a shared decoder should generate a distribution over future trajectories with a diffusion model, and the other should explicitly predict discrete control variables such as speed state. Both branches are trained together in a two-stage schedule, first learning perception, then freezing it while the hybrid decoder is learned. The claim matters because regression-only policies average away multiple valid driving options, while diffusion-only decoders sacrifice controllability and speed; the hybrid is proposed to keep both. If the reported results are correct, the approach is the top performer on the CARLA closed-loop benchmark and on NAVSIM among the compared methods. The practical consequence claimed is that a stochastic trajectory generator and a deterministic regulator can coexist in one network without slowing inference below real time.

What carries the argument

The central object is the hybrid diffusion-supervision decoder, a Transformer whose input token sequence is a concatenation of an encoded noised trajectory ($\ell_k$ tokens) and learned supervision queries ($\ell_s$ tokens), plus positional encodings. The sequence goes through self-attention, then cross-attention against a global condition vector built by concatenating multi-sensor fusion features, a target-goal embedding, and a diffusion-timestep embedding. After the decoder, the token stream is split at position $\ell_k$: the first part is passed to a diffusion head trained with a trajectory-reconstruction loss, the second to a supervision head for speed classification and auxiliary perception tasks. The same decoder is trained in two stages, with the perception module frozen in the second stage, and inference uses DDIM with two denoising steps.

What would settle it

Retrain DiffE2E and the published baselines on the exact official 36-route CARLA Longest6 split with identical weather, traffic density, and compute, and on NAVSIM with a shared image backbone (for example ResNet-34 instead of V2-99); if DiffE2E no longer tops the Driving Score and PDMS tables, the headline advantage is an artifact of protocol differences. A cheaper internal check is whether the reported one-stage-training collapse (DS 18.2) is reproducible, since that result carries much of the argument for the two-stage paradigm.

Watch

Extended reading notes

Core claim

On its own terms, the paper discovers that replacing the explicit-policy head with a diffusion head is not enough: a full Transformer decoder that jointly models noisy trajectories and supervised query tokens is what extracts the benefit. The experimental claim is that DiffE2E reaches a CARLA Longest6 Driving Score of 83, 13.7% above TF++WP and 20.3% above TF++, and a NAVSIM PDMS of 92.7, ahead of Hydra-MDP++ (91.0), GoalFlow (90.3), and DiffusionDrive (88.1). The attribution is architectural: the decoder's first block of tokens carries the diffusion latent for trajectory reconstruction, the second carries supervision latents for speed and auxiliary tasks, and cross-attention with fused perception, goal, and timestep conditions lets the two objectives reinforce each other. The paper also claims that 2 denoising steps suffice at inference, and that predicting noise instead of reconstructing the trajectory collapses the driving score from 82.9 to 20.1.

Load-bearing premise

The load-bearing premise is that the numbers in the CARLA and NAVSIM tables are directly comparable across papers, yet DiffE2E trains on a different 750k-frame expert dataset, evaluates on the six longest routes per town rather than the exact official split, and uses a larger image backbone than several baselines.

Editorial extensions

If this is right

  • A full diffusion decoder, rather than a diffusion head bolted onto a planning decoder, is a viable closed-loop policy and runs in 42.8 ms on one GPU with two denoising steps.
  • Anchored trajectory dictionaries and goal-conditioned flow matching are not required to win: DiffE2E outperforms DiffusionDrive and GoalFlow on NAVSIM while using a plain diffusion decoder with supervision.
  • Most of the margin over prior methods sits in safety sub-metrics (no-fault collision 99.9, time-to-collision 99.3), suggesting the hybrid objective improves the model's behavior in collision-prone interactions.
  • The output parametrization is decisive: trajectory reconstruction beats noise prediction by a large margin (82.9 vs 20.1 driving score), so future diffusion planners should frame the denoiser as a trajectory predictor, not an epsilon predictor.
  • The two-stage recipe is load-bearing: training perception and the hybrid decoder jointly from scratch drops the driving score to 18.2, implying that perception pretraining should be standard for diffusion-based end-to-end driving.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same split latent space may transfer to other embodied policy problems: a diffusion branch for multimodal action sequences plus a supervised branch for precise state variables (gripper pose, contact force, joint limits) could be dropped into manipulation or locomotion learning.
  • The reported leaderboard is not fully controlled, since baselines use different image encoders and the CARLA routes are the six longest routes per town rather than the official 36-route split; a controlled re-run could shrink or eliminate the margin.
  • The large gap between one-stage and two-stage training suggests that gradient flow through a perception encoder into a diffusion decoder is currently a bottleneck; one testable extension is to add a perceptual feature alignment loss or stop-gradient in one-stage training to close this gap without freezing.
  • Because the NAVSIM safety margin is concentrated in TTC and NC, isolating whether the diffusive stochasticity or the explicit supervision is responsible would be a targeted ablation: compare two-step stochastic decoding against deterministic decoding with the mode trajectory only.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes DiffE2E, an end-to-end driving policy that fuses camera and LiDAR features through hierarchical bidirectional cross-attention and decodes trajectories with a Transformer-based hybrid diffusion-supervision decoder. The decoder splits its latent space into a diffusion branch for trajectory distribution modeling and a supervised branch for auxiliary tasks such as speed classification, with cross-attention between the two. Training is performed in two stages on the TransFuser++ MPC dataset (CARLA) and the NAVSIM navtrain dataset, and the method is evaluated closed-loop in CARLA (Longest6, Town05 Long/Short) and non-reactively on NAVSIM Navtest. The paper claims state-of-the-art results on both platforms, with a DS of 83 on CARLA Longest6 and a PDMS of 92.7 on NAVSIM, and presents ablations showing the importance of each component and of the two-stage training scheme.

Significance. DiffE2E is a plausible architectural contribution: the hybrid diffusion-supervision decoder with structured latent splitting and the two-stage training paradigm are clearly described, and the internal ablations (Table 3) show that removing the ego state, command input, GRU, supervision branch, or the two-stage scheme degrades performance, which supports the hybrid design. The NAVSIM comparison is reasonably well controlled, since GoalFlow and Hydra-MDP++ also use the V2-99 backbone. However, the CARLA SOTA claim currently rests on a non-standard Longest6 protocol and is undermined by unresolved internal inconsistencies, so the empirical claims cannot yet be verified from the manuscript. If the protocol and consistency issues are fixed, the paper would be a solid empirical contribution to diffusion-based end-to-end driving.

major comments (4)
  1. [Appendix B.1.2 / Table 1] The Longest6 protocol described in Appendix B.1.2 is not the standard benchmark used by the baselines. The paper states it selects the 6 longest routes per town from the 76 training routes (36 routes total) and then "to increase evaluation difficulty" sets the highest traffic density, 6 weather conditions, 6 lighting conditions, and predefined adversarial scenarios. The published TransFuser++ and TransFuser++WP numbers in Table 1 were obtained under the standard Longest6 protocol with fixed weather and default traffic. Without route XMLs, an explicit weather/lighting condition list, episode count, traffic density setting, and random seeds, the claimed 13.7% and 20.3% DS improvements in Table 1 are not apples-to-apples. This is load-bearing for the central CARLA SOTA claim; the authors must either evaluate all baselines under the same custom protocol or report DiffE2E under the standard protocol.
  2. [Appendix A.1 / Section 4.1 / Appendix B.1.1] The two-stage training description is internally contradictory. Appendix A.1 states that the second stage "adopts an end-to-end joint training paradigm, simultaneously optimizing parameter distributions of both the perception module and diffusion decoder," whereas Section 4.1 and Appendix B.1.1 state that the perception module is frozen in the second stage ("the second stage freezes the trained perception module"). The one-stage vs. two-stage ablation in Table 3 (DS 18.2 vs. 82.9) is a central claim; the treatment of the perception module in stage two must be stated unambiguously to make the ablation reproducible and the comparison meaningful.
  3. [Section 3.2 / Table 3] Table 3 ablates a "GRU module" (w/o GRU, DS drops to 66.8), but no GRU is introduced in Section 3 or the appendix. The hybrid decoder is described as self-attention and cross-attention only, and the perception fusion module is also Transformer-based. Please specify where the GRU is used (e.g., temporal modeling of the fused features or the supervision head) and how it is integrated into the equations.
  4. [Tables 1, 6, 7 / Section 4.1] The CARLA results are reported as point estimates with no variance, number of episodes, or number of seeds. Given the stochasticity of CARLA and the custom weather/traffic settings in Appendix B.1.2, the reported differences (e.g., DS 73 vs. 83) cannot be statistically distinguished. Please report means and standard deviations over multiple evaluation runs or seeds.
minor comments (6)
  1. [Table 1] The TransFuser++WP row reads "C&L 73970.56" instead of "C&L 73 97 0.56"; the missing separators make the row unreadable.
  2. [Table 3] The row label "Full Discrimination" is unclear; it should be renamed to "Full Supervision" or "Explicit Policy" to contrast with "Full Diffusion."
  3. [Appendix B.1.1 / Table 4] The text says the first stage uses batch size 16 and the second stage uses 256, but Table 4 lists "Batch size (One-stage training) 256" and "Batch size (Two-stage training) 16"; these are inconsistent and should be corrected.
  4. [Appendix B.1.2] The description of "6 weather conditions" and "6 lighting conditions" is ambiguous, since CARLA weather presets jointly determine weather and lighting; a concrete list of the combined conditions should be provided.
  5. [Appendix C] The reference to "Transfuser[12]" should cite reference [9] (Transfuser), not [12] (NAVSIM).
  6. [Section D.2 / Table 8] The text states DiffE2E has "a relatively high parameter count (60M)", but Table 8 lists DiffE2E as 105M parameters; 60M is the parameter count of DiffusionDrive, so the sentence should refer to 105M.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are empirical benchmark comparisons with standard training losses, and no prediction reduces by construction to a fitted input.

full rationale

DiffE2E's central claims are closed-loop CARLA scores and NAVSIM PDMS numbers obtained by training a perception backbone and a hybrid diffusion-supervision decoder with standard losses (Eq. 9 trajectory reconstruction, Eq. 10 speed classification, and the multi-task supervision losses in Appendix A). None of these equations defines a predicted quantity in terms of the same quantity being fitted, and no ablation 'predicts' a value that is the direct output of its own training objective. The CARLA comparison against TF++ and TF++WP is an empirical leaderboard claim, not a derivation; the main risk is evaluation-protocol comparability (Appendix B.1.2 adds custom traffic density, weather/lighting conditions, and adversarial scenarios without releasing exact route XMLs or seeds), but that is a correctness and reproducibility concern, not circularity. The NAVSIM comparison is also empirical and uses the same V2-99 backbone as the closest baselines, so it is not circular. The paper contains internal inconsistencies—Table 3 ablates a 'GRU module' that is never introduced in Section 3, and Appendix A.1 describes the second training stage as jointly optimizing perception and decoder while Section 4.1 and Appendix B.1.1 state the perception module is frozen—but these flaws undermine verifiability rather than indicating that any result is equivalent to its inputs. There are no load-bearing self-citations: the cited prior works (TF++, Transfuser, DiffusionDrive, GoalFlow, Hydra-MDP++) are external baselines and backbones, not author-overlapping justifications. Appendix E explicitly acknowledges limitations in sampling efficiency and guidance, further indicating the paper does not rely on a self-validating derivation. Overall, the derivation chain is self-contained with respect to circularity, so the appropriate score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central claim rests mainly on engineering choices (denoising steps, loss weights, two-stage training) and on the assumption that the CARLA route setup is comparable to prior work. The paper invents no new physical entities. The free parameters are standard hyperparameters, but the denoising step count and loss weights are tuned on the evaluation benchmarks, which weakens the claim that the design is principled rather than fitted.

free parameters (4)
  • Denoising steps at inference = 2
    Chosen from the ablations in Figure 5 as the peak PDMS value. This is a hyperparameter fitted to the NAVSIM benchmark, not derived from theory.
  • Loss weights for supervision tasks = 1.0 or 10.0 depending on task
    Tables 4 and 5 list task-specific weights (e.g., diffusion 10.0, agent class 10.0, agent box 1.0 on NAVSIM) that are tuned to balance the multi-task loss.
  • Ego progress and TTC weights in PDMS comparison = 5.0 and 5.0
    These are PDMS weights inherited from NAVSIM, not introduced by the paper, but they are free parameters in the evaluation metric and affect the reported score.
  • Target point encoding dimension and positional encoding choices
    Dimensionalities such as d, ℓg, ℓt, ℓs are architectural choices not derived from first principles; they are tuned via experiments.
assumptions (4)
  • domain assumption The CARLA Longest6 evaluation setup used in this paper, with routes selected from the 76 training routes, is a valid proxy for the standard CARLA leaderboard Longest6 benchmark.
    Appendix B.1.2 describes a route selection procedure that differs from the official Longest6 protocol used by most cited baselines. If the routes or weather differ, the cross-method comparison in Table 1 is not directly valid.
  • domain assumption Diffusion trajectory reconstruction loss (Ldiff, Eq. 9) is an adequate training objective for closed-loop driving without additional safety constraints.
    The paper acknowledges in Limitations that trajectory generation may lack safety guidance. The assumption that simple reconstruction plus supervision suffices for safe closed-loop behavior is load-bearing for the claimed results.
  • domain assumption The TF++ MPC expert dataset (750k frames) is a representative and sufficient training distribution for the CARLA closed-loop benchmark.
    Appendix B.1.1 states the model is trained on this dataset. The benchmark routes overlap with the training route set (76 training routes), so generalization to truly unseen routes is not tested in the Longest6 setup.
  • ad hoc to paper Freezing the perception module and training only the diffusion decoder is a valid optimization strategy.
    The two-stage training is presented as a practical choice, and the ablation shows one-stage training fails (DS 18.2). This is an empirical design choice, not a theoretical guarantee, and it means the perception features are not jointly optimized for the planning objective.
invented entities (1)
  • Hybrid diffusion-supervision latent space (Zdiff and Zsup split)
    purpose: A structural decomposition of the decoder output into a diffusion trajectory stream and an explicit supervision stream, enabling joint training.
    This is a new architectural construct introduced by the paper, but it is an internal network design, not a new physical entity. Its efficacy is evidenced only by the paper's own benchmark results, which are not independently reproduced.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DiffE2E: Rethinking End-to-End Driving with a Hybrid Action Diffusion and Supervised Policy." pith.science (2026). https://pith.science/paper/RG4CVBGC

@misc{pith2026250519516,
  author       = {Pith},
  title        = {Pith review of: DiffE2E: Rethinking End-to-End Driving with a Hybrid Action Diffusion and Supervised Policy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RG4CVBGC}},
  note         = {Machine review of arXiv:2505.19516}
}
read the original abstract

End-to-end learning has emerged as a transformative paradigm in autonomous driving. However, the inherently multimodal nature of driving behaviors and the generalization challenges in long-tail scenarios remain critical obstacles to robust deployment. We propose DiffE2E, a diffusion-based end-to-end autonomous driving framework. This framework first performs multi-scale alignment of multi-sensor perception features through a hierarchical bidirectional cross-attention mechanism. It then introduces a novel class of hybrid diffusion-supervision decoders based on the Transformer architecture, and adopts a collaborative training paradigm that seamlessly integrates the strengths of both diffusion and supervised policy. DiffE2E models structured latent spaces, where diffusion captures the distribution of future trajectories and supervision enhances controllability and robustness. A global condition integration module enables deep fusion of perception features with high-level targets, significantly improving the quality of trajectory generation. Subsequently, a cross-attention mechanism facilitates efficient interaction between integrated features and hybrid latent variables, promoting the joint optimization of diffusion and supervision objectives for structured output generation, ultimately leading to more robust control. Experiments demonstrate that DiffE2E achieves state-of-the-art performance in both CARLA closed-loop evaluations and NAVSIM benchmarks. The proposed integrated diffusion-supervision policy offers a generalizable paradigm for hybrid action representation, with strong potential for extension to broader domains including embodied intelligence. More details and visualizations are available at \href{https://infinidrive.github.io/DiffE2E/}{project website}.

Figures

Figures reproduced from arXiv: 2505.19516 by the authors.

Figure 1
Figure 1. Comparison of end-to-end training paradigms. (a) Explicit Policy. Directly predicts trajectories through supervised learning after sensor input processing. (b) Explicit Policy with Diffusion Refinement. Uses diffusion models to replace traditional explicit policy trajectory output heads. (c) Diffusion Policy. Uses diffusion models to directly generate trajectories based on perception encoder features. amplify safety… view at source ↗
Figure 2
Figure 2. Overall architecture of DiffE2E. The main architecture consists of a Transformer-based perception module and a Hybrid Diffusion and Supervision Decoder. The blue arrows (→) indicate the data flow exclusively used for the CARLA benchmark, while the black arrows (→) represent the data flow shared between both the CARLA and NAVSIM benchmarks. 3.1 Multimodal Fusion Perception Module The perception module aims to fuse mu… view at source ↗
Figure 3
Figure 3. Visualization in CARLA Simulator. In both the LiDAR and scene visualizations, blue points represent the predicted trajectory, while red points in the LiDAR view denote the target waypoints. and DiffusionDrive (88.1). This highlights the strength of our diffusion-based end-to-end approach in multi-dimensional driving evaluation. In safety and compliance, DiffE2E excels: it achieves a no-fault collision rate of 99.9 (… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Visualization in Navtest benchmark. Red trajectories denote the predicted paths of each method, while green trajectory corresponds to the ground truth. demonstrates DiffE2E’s accuracy and safety in trajectory planning. More visualization results can be found in Section…
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Visualization on Navtest benchmark. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Visualization on Navtest benchmark. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OmniNWM: Omniscient Driving Navigation World Models

    cs.CV 2025-10 conditional novelty 6.0 of 10

    OmniNWM jointly generates long panoramic multi-modal driving videos, controls them precisely via normalized Plücker ray-maps, and derives dense driving rewards from generated 3D occupancy.

Reference graph

Works this paper leans on

62 extracted references · 14 canonical work pages · cited by 1 Pith paper

  1. [1]

    nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles.arXiv preprint arXiv:2106.11810, 2021

    Holger Caesar, Juraj Kabzan, Kok Seang Tan, Whye Kit Fong, Eric Wolff, Alex Lang, Luke Fletcher, Oscar Beijbom, and Sammy Omari. nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles.arXiv preprint arXiv:2106.11810, 2021

    arXiv 2021

  2. [2]

    Learning from all vehicles

    Dian Chen and Philipp Kr¨ahenb¨uhl. Learning from all vehicles. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17222–17231, 2022

    work page 2022

  3. [3]

    Learning by cheating

    Dian Chen, Brady Zhou, Vladlen Koltun, and Philipp Kr¨ahenb¨uhl. Learning by cheating. In Conference on Robot Learning, pages 66–75. PMLR, 2020

    work page 2020

  4. [4]

    Learning to drive from a world on rails

    Dian Chen, Vladlen Koltun, and Philipp Kr ¨ahenb¨uhl. Learning to drive from a world on rails. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 15590–15599, 2021

    work page 2021

  5. [5]

    End-to-end autonomous driving: Challenges and frontiers.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, and Hongyang Li. End-to-end autonomous driving: Challenges and frontiers.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  6. [6]

    Vadv2: End-to-end vectorized autonomous driving via probabilistic planning.arXiv preprint arXiv:2402.13243, 2024

    Shaoyu Chen, Bo Jiang, Hao Gao, Bencheng Liao, Qing Xu, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Vadv2: End-to-end vectorized autonomous driving via probabilistic planning.arXiv preprint arXiv:2402.13243, 2024

    arXiv 2024

  7. [7]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, page 02783649241273668, 2023

    2023

  8. [8]

    Neat: Neural attention fields for end- to-end autonomous driving

    Kashyap Chitta, Aditya Prakash, and Andreas Geiger. Neat: Neural attention fields for end- to-end autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 15793–15803, 2021

    2021

Show all 62 references
  1. [9]

    Transfuser: Imitation with transformer-based sensor fusion for autonomous driving.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(11):12878–12895, 2022

    Kashyap Chitta, Aditya Prakash, Bernhard Jaeger, Zehao Yu, Katrin Renz, and Andreas Geiger. Transfuser: Imitation with transformer-based sensor fusion for autonomous driving.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(11):12878–12895, 2022

    2022

  2. [10]

    Exploring the limitations of behavior cloning for autonomous driving

    Felipe Codevilla, Eder Santana, Antonio M L´opez, and Adrien Gaidon. Exploring the limitations of behavior cloning for autonomous driving. InProceedings of the IEEE/CVF international conference on computer vision, pages 9329–9338, 2019

    2019

  3. [11]

    Openscene: The largest up-to-date 3d occupancy prediction bench- mark in autonomous driving.https://github.com/OpenDriveLab/OpenScene, 2023

    OpenScene Contributors. Openscene: The largest up-to-date 3d occupancy prediction bench- mark in autonomous driving.https://github.com/OpenDriveLab/OpenScene, 2023

    2023

  4. [12]

    Navsim: Data-driven non- reactive autonomous vehicle simulation and benchmarking.Advances in Neural Information Processing Systems, 37:28706–28719, 2024

    Daniel Dauner, Marcel Hallgarten, Tianyu Li, Xinshuo Weng, Zhiyu Huang, Zetong Yang, Hongyang Li, Igor Gilitschenski, Boris Ivanovic, Marco Pavone, et al. Navsim: Data-driven non- reactive autonomous vehicle simulation and benchmarking.Advances in Neural Information Processing...

    2024

  5. [13]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021

    2021

  6. [14]

    Carla: An open urban driving simulator

    Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. InConference on robot learning, pages 1–16. PMLR, 2017. 10

    2017

  7. [15]

    One step diffusion via shortcut models.arXiv preprint arXiv:2410.12557, 2024

    Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models.arXiv preprint arXiv:2410.12557, 2024

    2024 arXiv

  8. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  9. [17]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  10. [18]

    Model-based imitation learning for urban driving.Advances in Neural Information Processing Systems, 35:20703–20716, 2022

    Anthony Hu, Gianluca Corrado, Nicolas Griffiths, Zachary Murez, Corina Gurau, Hudson Yeo, Alex Kendall, Roberto Cipolla, and Jamie Shotton. Model-based imitation learning for urban driving.Advances in Neural Information Processing Systems, 35:20703–20716, 2022

    2022

  11. [19]

    St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning

    Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, and Dacheng Tao. St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning. InEuropean Conference on Computer Vision, pages 533–549. Springer, 2022

    2022

  12. [20]

    Planning-oriented autonomous driving

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17853–17862, 2023

    2023

  13. [21]

    Versatile behavior diffusion for generalized traffic agent simulation.arXiv preprint arXiv:2404.02524, 2024

    Zhiyu Huang, Zixu Zhang, Ameya Vaidya, Yuxiao Chen, Chen Lv, and Jaime Fern ´andez Fisac. Versatile behavior diffusion for generalized traffic agent simulation.arXiv preprint arXiv:2404.02524, 2024

    2024

  14. [22]

    Hidden biases of end-to-end driving models

    Bernhard Jaeger, Kashyap Chitta, and Andreas Geiger. Hidden biases of end-to-end driving models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 8240–8249, 2023

    2023

  15. [23]

    Planning with diffusion for flexible behavior synthesis.arXiv preprint arXiv:2205.09991, 2022

    Michael Janner, Yilun Du, Joshua B Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis.arXiv preprint arXiv:2205.09991, 2022

    2022 arXiv

  16. [24]

    Driveadapter: Breaking the coupling barrier of perception and planning in end-to-end au- tonomous driving

    Xiaosong Jia, Yulu Gao, Li Chen, Junchi Yan, Patrick Langechuan Liu, and Hongyang Li. Driveadapter: Breaking the coupling barrier of perception and planning in end-to-end au- tonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 795...

    2023

  17. [25]

    Think twice before driving: Towards scalable decoders for end-to-end autonomous driving

    Xiaosong Jia, Penghao Wu, Li Chen, Jiangwei Xie, Conghui He, Junchi Yan, and Hongyang Li. Think twice before driving: Towards scalable decoders for end-to-end autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21983–...

    2023

  18. [26]

    Vad: Vectorized scene representation for efficient autonomous driving

    Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene representation for efficient autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pag...

    2023

  19. [27]

    Motiondiffuser: Controllable multi-agent motion prediction using diffusion

    Chiyu Jiang, Andre Cornman, Cheolho Park, Benjamin Sapp, Yin Zhou, Dragomir Anguelov, et al. Motiondiffuser: Controllable multi-agent motion prediction using diffusion. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9644–9653, 2023

    2023

  20. [28]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. InProc. NeurIPS, 2022

    2022

  21. [29]

    An energy and gpu-computation efficient backbone network for real-time object detection

    Youngwan Lee, Joong-won Hwang, Sangrok Lee, Yuseok Bae, and Jongyoul Park. An energy and gpu-computation efficient backbone network for real-time object detection. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 0–0, 2019. 11

    2019

  22. [30]

    Hydra-mdp++: Advancing end-to-end driving via expert-guided hydra-distillation.arXiv preprint arXiv:2503.12820, 2025

    Kailin Li, Zhenxin Li, Shiyi Lan, Yuan Xie, Zhizhong Zhang, Jiayi Liu, Zuxuan Wu, Zhiding Yu, and Jose M Alvarez. Hydra-mdp++: Advancing end-to-end driving via expert-guided hydra-distillation.arXiv preprint arXiv:2503.12820, 2025

    2025 arXiv

  23. [31]

    Enhancing end-to-end autonomous driving with latent world model.arXiv preprint arXiv:2406.08481, 2024

    Yingyan Li, Lue Fan, Jiawei He, Yuqi Wang, Yuntao Chen, Zhaoxiang Zhang, and Tieniu Tan. Enhancing end-to-end autonomous driving with latent world model.arXiv preprint arXiv:2406.08481, 2024

    2024 arXiv

  24. [32]

    Hydra-mdp: End-to-end multimodal planning with multi-target hydra-distillation.arXiv preprint arXiv:2406.06978, 2024

    Zhenxin Li, Kailin Li, Shihao Wang, Shiyi Lan, Zhiding Yu, Yishen Ji, Zhiqi Li, Ziyue Zhu, Jan Kautz, Zuxuan Wu, et al. Hydra-mdp: End-to-end multimodal planning with multi-target hydra-distillation.arXiv preprint arXiv:2406.06978, 2024

    2024 arXiv

  25. [33]

    Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving.arXiv preprint arXiv:2411.15139, 2024

    Bencheng Liao, Shaoyu Chen, Haoran Yin, Bo Jiang, Cheng Wang, Sixu Yan, Xinbang Zhang, Xiangyu Li, Ying Zhang, Qian Zhang, et al. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving.arXiv preprint arXiv:2411.15139, 2024

    2024 arXiv

  26. [34]

    Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022

    2022 arXiv

  27. [35]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps.Advances in Neural Information Processing Systems, 35:5775–5787, 2022

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps.Advances in Neural Information Processing Systems, 35:5775–5787, 2022

    2022

  28. [36]

    On distillation of guided diffusion models

    Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14297–14306, 2023

    2023

  29. [37]

    Multi-modal fusion transformer for end-to-end autonomous driving

    Aditya Prakash, Kashyap Chitta, and Andreas Geiger. Multi-modal fusion transformer for end-to-end autonomous driving. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7077–7087, 2021

    2021

  30. [38]

    Design- ing network design spaces

    Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Doll´ar. Design- ing network design spaces. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10428–10436, 2020

    2020

  31. [39]

    Plant: Explainable planning transformers via object-level representations.arXiv preprint arXiv:2210.14222, 2022

    Katrin Renz, Kashyap Chitta, Otniel-Bogdan Mercea, A Koepke, Zeynep Akata, and Andreas Geiger. Plant: Explainable planning transformers via object-level representations.arXiv preprint arXiv:2210.14222, 2022

    2022 arXiv

  32. [40]

    Motionlm: Multi-agent motion forecasting as language modeling

    Ari Seff, Brian Cera, Dian Chen, Mason Ng, Aurick Zhou, Nigamaa Nayakanti, Khaled S Refaat, Rami Al-Rfou, and Benjamin Sapp. Motionlm: Multi-agent motion forecasting as language modeling. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 8579–8590, 2023

    2023

  33. [41]

    Safety-enhanced au- tonomous driving using interpretable sensor fusion transformer

    Hao Shao, Letian Wang, Ruobing Chen, Hongsheng Li, and Yu Liu. Safety-enhanced au- tonomous driving using interpretable sensor fusion transformer. InConference on Robot Learning, pages 726–737. PMLR, 2023

    2023

  34. [42]

    Reasonnet: End-to-end driving with temporal and global reasoning

    Hao Shao, Letian Wang, Ruobing Chen, Steven L Waslander, Hongsheng Li, and Yu Liu. Reasonnet: End-to-end driving with temporal and global reasoning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13723–13733, 2023

    2023

  35. [43]

    Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  36. [44]

    Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020

    2011 arXiv

  37. [45]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. 2023

    2023

  38. [46]

    Sparsedrive: End-to-end autonomous driving via sparse scene representation.arXiv preprint arXiv:2405.19620, 2024

    Wenchao Sun, Xuewu Lin, Yining Shi, Chuang Zhang, Haoran Wu, and Sifa Zheng. Sparsedrive: End-to-end autonomous driving via sparse scene representation.arXiv preprint arXiv:2405.19620, 2024. 12

    2024 arXiv

  39. [47]

    A survey of end-to-end driving: Architectures and training methods.IEEE Transactions on Neural Networks and Learning Systems, 33(4):1364–1384, 2020

    Ardi Tampuu, Tambet Matiisen, Maksym Semikin, Dmytro Fishman, and Naveed Muhammad. A survey of end-to-end driving: Architectures and training methods.IEEE Transactions on Neural Networks and Learning Systems, 33(4):1364–1384, 2020

    2020

  40. [48]

    Efficientnet: Rethinking model scaling for convolutional neural networks

    Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. InInternational conference on machine learning, pages 6105–6114. PMLR, 2019

    2019

  41. [49]

    He-drive: Human-like end-to-end driving with vision language models.arXiv preprint arXiv:2410.05051, 2024

    Junming Wang, Xingyu Zhang, Zebin Xing, Songen Gu, Xiaoyang Guo, Yang Hu, Ziying Song, Qian Zhang, Xiaoxiao Long, and Wei Yin. He-drive: Human-like end-to-end driving with vision language models.arXiv preprint arXiv:2410.05051, 2024

  42. [50]

    Drivemlm: Aligning multi-modal large language models with behavioral planning states for autonomous driving.arXiv preprint arXiv:2312.09245, 2023

    Wenhai Wang, Jiangwei Xie, ChuanYang Hu, Haoming Zou, Jianan Fan, Wenwen Tong, Yang Wen, Silei Wu, Hanming Deng, Zhiqi Li, et al. Drivemlm: Aligning multi-modal large language models with behavioral planning states for autonomous driving.arXiv preprint arXiv:2312.09245, 2023

    2023

  43. [51]

    Para-drive: Par- allelized architecture for real-time autonomous driving

    Xinshuo Weng, Boris Ivanovic, Yan Wang, Yue Wang, and Marco Pavone. Para-drive: Par- allelized architecture for real-time autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15449–15458, 2024

    2024

  44. [52]

    Trajectory-guided control prediction for end-to-end autonomous driving: A simple yet strong baseline.Advances in Neural Information Processing Systems, 35:6119–6132, 2022

    Penghao Wu, Xiaosong Jia, Li Chen, Junchi Yan, Hongyang Li, and Yu Qiao. Trajectory-guided control prediction for end-to-end autonomous driving: A simple yet strong baseline.Advances in Neural Information Processing Systems, 35:6119–6132, 2022

    2022

  45. [53]

    Goalflow: Goal-driven flow matching for multimodal trajectories generation in end-to-end autonomous driving.arXiv preprint arXiv:2503.05689, 2025

    Zebin Xing, Xingyu Zhang, Yang Hu, Bo Jiang, Tong He, Qian Zhang, Xiaoxiao Long, and Wei Yin. Goalflow: Goal-driven flow matching for multimodal trajectories generation in end-to-end autonomous driving.arXiv preprint arXiv:2503.05689, 2025

    2025

  46. [54]

    Diffusion-es: Gradient-free planning with diffusion for autonomous driving and zero-shot instruction following.arXiv preprint arXiv:2402.06559, 2024

    Brian Yang, Huangyuan Su, Nikolaos Gkanatsios, Tsung-Wei Ke, Ayush Jain, Jeff Schneider, and Katerina Fragkiadaki. Diffusion-es: Gradient-free planning with diffusion for autonomous driving and zero-shot instruction following.arXiv preprint arXiv:2402.06559, 2024

    2024 arXiv

  47. [55]

    Drama: An efficient end-to-end motion planner for autonomous driving with mamba.arXiv preprint arXiv:2408.03601, 2024

    Chengran Yuan, Zhanqi Zhang, Jiawei Sun, Shuo Sun, Zefan Huang, Christina Dao Wen Lee, Dongen Li, Yuhang Han, Anthony Wong, Keng Peng Tee, et al. Drama: An efficient end-to-end motion planner for autonomous driving with mamba.arXiv preprint arXiv:2408.03601, 2024

    2024 arXiv

  48. [56]

    3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations.arXiv preprint arXiv:2403.03954, 2024

    Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations.arXiv preprint arXiv:2403.03954, 2024

    2024 arXiv

  49. [57]

    Rethinking the open-loop evaluation of end-to-end autonomous driving in nuscenes.arXiv preprint arXiv:2305.10430, 2023

    Jiang-Tian Zhai, Ze Feng, Jinhao Du, Yongqiang Mao, Jiang-Jiang Liu, Zichang Tan, Yifu Zhang, Xiaoqing Ye, and Jingdong Wang. Rethinking the open-loop evaluation of end-to-end autonomous driving in nuscenes.arXiv preprint arXiv:2305.10430, 2023

    2023 arXiv

  50. [58]

    Scaling vision transform- ers

    Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transform- ers. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12104–12113, 2022

    2022

  51. [59]

    End-to-end urban driving by imitating a reinforcement learning coach

    Zhejun Zhang, Alexander Liniger, Dengxin Dai, Fisher Yu, and Luc Van Gool. End-to-end urban driving by imitating a reinforcement learning coach. InProceedings of the IEEE/CVF international conference on computer vision, pages 15222–15232, 2021

    2021

  52. [60]

    Diffusion-based planning for autonomous driving with flexible guidance.arXiv preprint arXiv:2501.15564, 2025

    Yinan Zheng, Ruiming Liang, Kexin Zheng, Jinliang Zheng, Liyuan Mao, Jianxiong Li, Weihao Gu, Rui Ai, Shengbo Eben Li, Xianyuan Zhan, et al. Diffusion-based planning for autonomous driving with flexible guidance.arXiv preprint arXiv:2501.15564, 2025

    2025 arXiv

  53. [61]

    world-on-rails

    Julian Zimmerlin, Jens Beißwenger, Bernhard Jaeger, Andreas Geiger, and Kashyap Chitta. Hidden biases of end-to-end driving datasets.arXiv preprint arXiv:2412.09602, 2024. 13 A Details of Multi-task Loss Function Design A.1 Loss Function Design Compared to the deterministic ma...

    2024 arXiv

  54. [62]

    This enormous difference reveals that directly modeling the trajectory space is more effective than the noise space for tasks requiring high precision, such as autonomous driving

    The results were surprising—when the model outputs noise, the driving score plummets to 20.1, a staggering 76% decrease compared to outputting trajectories, with vehicles barely able to follow lanes in the simulation environment. This enormous difference reveals that directly ...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.