Pith. sign in

REVIEW 4 major objections 6 minor 5 cited by

Challenger: Affordable Adversarial Driving Video Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Challenger is a framework that automatically converts ordinary real-world driving scenes into photorealistic adversarial videos—cut-ins, sudden lane changes, tailgating, and blind-spot intrusions—that raise collision rates of four…

desk verdict A solid and useful capability paper whose empirical results look real, but the trajectory scorer's normalized challenge term doesn't implement the advertised closeness proxy — worth careful review, not dismissal. read the letter →

arxiv 2505.15880 v2 pith:VYWMIAYV submitted 2025-05-21 cs.CV

Zhiyuan Xu , Bohan Li , Huan-ang Gao , Mingju Gao , Yong Chen , Ming Liu , Chenxu Yan , Hang Zhao
show 2 more authors
Shuo Feng Hao Zhao
This is my paper
classification cs.CV
keywords autonomousdrivingadversarialscenariogenerationvideodiffusionmodelsend-to-endplanningsafetyevaluationmultiviewrenderingnuScenes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Challenger aims to fill a gap: no existing method automatically produces diverse, physically plausible, and photorealistic adversarial driving videos inside real-world scenes. It differs from prior work by decoupling adversarial trajectory search from video synthesis—candidate maneuvers are sampled from a diffusion model, simulated through a physics-aware planner, scored for drivability, collision avoidance, and challenge to the ego vehicle, then refined over multiple rounds before any rendering happens. On the nuScenes validation set the framework builds a 156-scene corpus called Adv-nuSc, and the authors report that four state-of-the-art end-to-end AD models collide substantially more often on it (for example, UniAD's average collision rate rises from 0.29% to 3.95%, and VAD's from 0.26% to 7.05%). The paper also reports that adversarial maneuvers selected using one model often transfer to other models, suggesting shared vulnerabilities across architectures. If correct, the result makes sensor-level adversarial stress-testing of autonomous driving systems scalable and automated rather than handcrafted.

What carries the argument

The load-bearing mechanism is the multi-round trajectory refinement loop, driven by a diffusion-based trajectory generator and a physics-aware planning simulator. Candidate waypoint trajectories $\tau \in \mathbb{R}^{B \times T \times 2}$ are sampled from an unconditional diffusion model, tracked by an LQR controller, and propagated with a kinematic bicycle model; the resulting 3D bounding boxes are scored by a weighted function whose adversarial-challenge term rewards close encounters with the ego vehicle. Top-scoring trajectories are resampled with probability proportional to the softmax of their scores, perturbed with Gaussian noise, and denoised through a truncated reverse diffusion process, and the cycle repeats for several rounds. This loop makes adversarial search affordable because it operates on abstract trajectories and never renders candidate videos; rendering happens once, at the end, with a multiview diffusion renderer.

What would settle it

Take the Adv-nuSc scenes and replace each adversarial vehicle's maneuver with a benign lane-following motion that keeps the same distance from the ego car, then re-run the same four end-to-end planners; if collision rates stay high, closeness alone explains the failures, and if they fall sharply, the aggressive maneuver content chosen by the scorer is the active ingredient.

Watch

Extended reading notes

Core claim

The paper's central claim is that adversarial driving videos—photorealistic, multiview, physically plausible, and deliberately challenging—can be generated automatically and affordably by separating the search for adversarial trajectories from the expensive rendering step. The mechanism is a loop that repeatedly samples candidate trajectories from a diffusion model trained on naturalistic driving data, simulates them through an LQR-tracked kinematic bicycle model, scores them with a weighted combination of drivable-area compliance, collision avoidance, and adversarial challenge, then resamples, adds noise, and denoises the best candidates over several rounds. After the search converges, the top trajectory is applied to a background vehicle's 3D bounding boxes and the updated scene is rendered with a multiview diffusion renderer. The Adv-nuSc dataset built this way raises collision rates of four end-to-end planners by large margins, and scenarios filtered by one model's failures transfer to the other models tested. The paper therefore claims to provide the first automatic, scalable source of sensor-level adversarial scenarios for evaluating end-to-end driving systems.

Load-bearing premise

Everything rests on the scorer's assumption that the most adversarial trajectory is the one that brings the adversarial vehicle geometrically close to the ego car without colliding; if closeness to the ego is not what breaks end-to-end planners, the reported collision-rate increases could shrink or disappear.

Editorial extensions

If this is right

  • Adversarial scenario generation for vision-based end-to-end AD evaluation becomes automatic and scalable: because the expensive video rendering happens only after trajectory search, a large corpus like Adv-nuSc (156 scenes, 6,115 samples) can be produced without rendering every candidate trajectory.
  • Because scenarios filtered by one model's failures transfer to other models, a single surrogate planner can build a stress-test corpus that exposes common weaknesses across different end-to-end architectures.
  • The generated scenarios are kept challenging yet solvable—scenes are discarded if the adversarial vehicle collides or if the ego vehicle cannot safely follow its ground-truth trajectory—so the corpus can serve as an evaluation set rather than only an attack.
  • The open-loop, vision-only evaluation setting means the same recipe can be extended to closed-loop simulation or to models consuming LiDAR or HD maps, directions the paper explicitly lists as future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A consequence the authors leave implicit: if geometric closeness to the ego vehicle is a valid proxy for adversarialness, the scorer will systematically miss failure modes that do not involve close approaches—such as perceptual edge cases or appearance-based attacks—so the generated corpus covers only one class of vulnerabilities.
  • A testable extension: train an end-to-end model on Adv-nuSc videos together with the original nuScenes data and measure whether collision rates on ordinary scenes stay flat while adversarial collision rates drop; that would indicate whether the generated failures are addressable through training rather than requiring architectural changes.
  • The paper's evaluation is open-loop, so the natural next stress test is to feed the generated scenes into a closed-loop simulator and see whether the collision-rate increase persists when the ego vehicle can react to the adversarial maneuver in real time.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces Challenger, a framework for generating photorealistic, multiview driving videos in which a background vehicle is reprogrammed to execute adversarial maneuvers (cut-ins, tailgating, lane blocking, blind-spot intrusions). The pipeline combines an unconditional diffusion model for trajectory proposals, a physics-aware simulator (LQR controller plus kinematic bicycle model), a composite trajectory scorer with terms for drivable-area compliance, collision avoidance, and adversarial challenge, and a multi-round refinement scheme inspired by Diffusion-ES. A multiview neural renderer (MagicDriveDiT) produces the final videos. The authors construct an Adv-nuSc dataset from the nuScenes validation set and report that four end-to-end AD models (UniAD, VAD, SparseDrive, DiffusionDrive) exhibit strongly increased collision rates on Adv-nuSc relative to nuScenes and to a re-rendered nuScenes-val-R, with attack transferability across models. An ablation study attributes much of the effect to the trajectory scorer and the multi-round refinement process.

Significance. If the results hold, Challenger is a valuable step toward scalable, automated generation of sensor-level adversarial scenarios for stress-testing end-to-end AD systems. The paper releases code, models, and the Adv-nuSc dataset, and it evaluates four state-of-the-art AD models, including a transferability analysis that is often missing in this literature. The authors are transparent about hyperparameters and provide a runtime breakdown, which is useful for reproducibility. The central measured effect, a substantial increase in collision rates on rendered adversarial videos relative to both real and re-rendered non-adversarial data, is plausible and clearly described. However, the paper's explanation of the adversarial mechanism rests on a trajectory-scoring term whose normalization appears to discard absolute proximity to the ego vehicle, and the quantitative evaluation lacks uncertainty measures and a comparison against simple heuristic baselines. These issues need to be resolved before the claims can be fully accepted.

major comments (4)
  1. [Section 3.3 / Appendix A.1] The adversarial challenge score is defined in Appendix A.1 as ACScore = (ACScore' - min(ACScore')) / (max(ACScore') - min(ACScore')), where ACScore' = Dist * ACStepWeight and Dist is the per-timestep center-to-center distance vector. If, as the notation suggests, the min and max are taken over the H timesteps of each individual trajectory, then the score is invariant to adding a constant to all entries of Dist and to scaling Dist by a positive factor. In other words, it measures the shape of the distance profile rather than absolute closeness to the ego vehicle. This contradicts Section 3.3, which states that the score 'rewards close encounters with the ego vehicle,' and it weakens the interpretation of the ablation in Table 2, where removing the trajectory scorer collapses the failure rate from 6.17% to 0.13%. The active ingredient in the scorer could be the decreasing-distance trend or the other terms (drivable-area compliance, collision penalty) rather than absolute proximity. Please reformulate the score to include absolute distance (for example, by normalizing against a fixed global reference or by adding a distance penalty), or, if the intended behavior is indeed to reward relative approach, revise the description and provide an ablation that isolates the adversarial-challenge term and measures the actual proximity of the selected trajectories.
  2. [Section 4.5 / Table 2] The ablation study is based on 20 randomly selected scenes, and the reported percentages are not accompanied by confidence intervals or statistical tests. The text states that the metric is 'the percentage of samples that Challenger could successfully generate while the representative UniAD model failed to navigate safely,' but it is not clear whether the denominator is all attempted scenes, all successfully generated scenes, or all scenes evaluated by UniAD. With only 20 scenes, the difference between 6.17% and 3.49% for the 'Challenger' versus '- MTR' configurations may not be significant, and the near-zero value for '- TS' could reflect an inability to generate any scenes rather than an inability to induce failures. Please report per-scene results, bootstrap confidence intervals or equivalent, and a precise definition of the denominator.
  3. [Section 4.2 / Table 1] The main evaluation reports collision rates without any measure of uncertainty or significance testing. The baseline rates are very small (for example, UniAD average 0.29%, SparseDrive average 0.107%), so the large fold increases on Adv-nuSc could in principle be driven by a small number of scenes or by scene-selection effects. Since the same underlying nuScenes scenes are used for nuScenes, nuScenes-val-R, and Adv-nuSc, please provide per-scene variability, confidence intervals, and a paired statistical test (or a comparable analysis) to establish that the increases are not an artifact of the particular scenes retained in Adv-nuSc.
  4. [Sections 4.2 and 4.5] The paper does not compare Challenger against a simple heuristic adversarial planner (for example, rule-based cut-in or tailgating maneuvers) or against random trajectory sampling with the same rendering pipeline. Without such baselines, it is difficult to attribute the observed collision-rate increases to the multi-round refinement and trajectory scoring rather than to the mere presence of an aggressive agent in the scene or to the renderer. The open-loop evaluation, which the authors acknowledge in Section 6, also limits the practical significance; a closed-loop extension or an explicit discussion of the challenges and possible remedies would strengthen the paper.
minor comments (6)
  1. [Appendix A.1] The text defines ACStepWeight as a vector in R^T, but the planning horizon is denoted H elsewhere in the same section; please make the notation consistent.
  2. [Algorithm 1] The truncated diffusion step uses the notation sqrt(alpha_ttrunc) without defining ttrunc or the alpha schedule; please specify the truncated reverse process in the text or in the algorithm caption.
  3. [Section 4.5] The phrase 'the percentage of samples that Challenger could successfully generate while the representative UniAD model failed to navigate safely' is ambiguous; please clarify the denominator and state whether non-generated samples are counted as failures or excluded.
  4. [Table 4] The planning horizon H is given as 12 timesteps, but the timestep duration is not stated; since nuScenes annotations are at 2 Hz, please confirm whether 12 timesteps correspond to 6 seconds and state this explicitly.
  5. [Appendix D] The filtering step discards scenes where the adversarial vehicle 'never gets close to the ego vehicle (e.g., never appears directly in front, behind, or beside it at any time),' but no distance or geometry threshold is specified; please define this criterion quantitatively.
  6. [Figure 4] The caption states that the figure shows 'failure rates of victim models,' but the y-axis label is missing from the figure; please add axis labels and units.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the adversarial trajectory scorer is an independent proximity heuristic, and the collision-rate evaluation is measured from the AD models' own planned trajectories, not fitted into the generator; the only mild concern is that the Adv-nuSc test set is filtered and ranked by the same challenge score used during generation.

full rationale

The central claim is that Challenger-generated videos increase the collision rates of four end-to-end AD models. The generation pipeline optimizes a hand-designed trajectory scorer (drivable-area compliance, collision penalty, and an adversarial-challenge term based on center-to-center distance to the ego), not the AD models' collision rates. The reported evaluation metric is computed from the AD models' own planned trajectories on rendered multiview videos, so the headline collision-rate increases are not defined into the generator by construction. The ablation in Table 2 shows that removing the trajectory scorer (TS) collapses the failure rate from 6.17% to 0.13%, but that is evidence that the proxy is causally useful, not evidence that the target metric is fitted. Self-citations appear in the related-work survey (e.g., INT2, MARS, P-MapNet, DiST-4D, HINT-AD) but are not load-bearing for any derivation in the method or evaluation. One mild, non-circular concern is that Appendix D builds Adv-nuSc by restricting to vehicles that come close to the ego and ranking scenes by the same adversarial-challenge score used in Section 3.3, so the evaluation set is enriched for high-proxy scenarios; this is test-set selection rather than a derivation circle, because the target metric (AD collision) is measured independently. A separate validity caveat, also not a circularity, is that the min-max normalization in Appendix A.1 is applied per trajectory and therefore discards absolute distance, so the 'close encounters' wording overstates what the score strictly optimizes; this affects the strength of the proxy, not the independence of the collision evaluation.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The system does not introduce new physical entities. The free parameters are hand-set scoring and search hyperparameters, not fitted to the evaluation data. The principled assumptions are the realism of the trained diffusion trajectory model, the adequacy of bicycle-model dynamics, the fidelity of MagicDriveDiT rendering, and the validity of the proximity-based adversarial proxy. The last assumption is the most consequential because the ablation shows the scorer is responsible for the collision-rate increase.

free parameters (8)
  • DACWeight = 5
    Scoring weight for drivable area compliance, hand-chosen (Table 4).
  • CRWeight = 10
    Scoring weight for collision rate, hand-chosen (Table 4).
  • ACWeight = 1
    Scoring weight for adversarial challenge, hand-chosen (Table 4).
  • CornerWeight = 0.2
    Penalty ratio between center and corner drivable-area violations (Table 4).
  • InitScale = 1e-3
    Random score initialization scale (Table 4).
  • Temperature = 1.0
    Resampling softmax temperature in multi-round refinement (Table 4).
  • H = 12
    Planning horizon in timesteps (Table 4).
  • #Refinement Rounds = 5
    Number of refinement rounds (Table 4).
assumptions (5)
  • domain assumption Kinematic bicycle model and LQR controller produce physically feasible trajectories for arbitrary vehicle sizes in nuScenes.
    Invoked in Section 3.2; the diffusion model outputs are not guaranteed feasible for the selected vehicle, so the simulator is used to enforce dynamics, assuming bicycle-model dynamics are adequate.
  • domain assumption An unconditional diffusion model trained on nuPlan trajectories captures the distribution of naturalistic driving behavior sufficiently to generate and mutate plausible trajectories in nuScenes scenes.
    Section 3.1; no quantitative validation of trajectory realism is provided.
  • domain assumption MagicDriveDiT, conditioned on BEV maps and 3D boxes, can render photorealistic multiview videos of edited adversarial scenes.
    Section 3.5; the renderer is treated as a black box and its failures on aggressive maneuvers are only partially assessed via SC/IQ metrics.
  • ad hoc to paper Adversarial challenge is adequately measured by center-to-center distance to the ego vehicle.
    Section 3.3 and Appendix D; this proxy is the core of the scoring function and the dataset filtering, and is not validated against an independent measure of model failure on a development set.
  • domain assumption Grounded-SAM 2 segmentation, filtered by projected 3D boxes, gives a valid multi-subject video quality estimate.
    Appendix C; used to report subject consistency and imaging quality, but the adapted VBench protocol is not validated against human judgments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Challenger: Affordable Adversarial Driving Video Generation." pith.science (2026). https://pith.science/paper/VYWMIAYV

@misc{pith2026250515880,
  author       = {Pith},
  title        = {Pith review of: Challenger: Affordable Adversarial Driving Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VYWMIAYV}},
  note         = {Machine review of arXiv:2505.15880}
}
read the original abstract

Generating photorealistic driving videos has seen significant progress recently, but current methods largely focus on ordinary, non-adversarial scenarios. Meanwhile, efforts to generate adversarial driving scenarios often operate on abstract trajectory or BEV representations, falling short of delivering realistic sensor data that can truly stress-test autonomous driving (AD) systems. In this work, we introduce Challenger, a framework that produces physically plausible yet photorealistic adversarial driving videos. Generating such videos poses a fundamental challenge: it requires jointly optimizing over the space of traffic interactions and high-fidelity sensor observations. Challenger makes this affordable through two techniques: (1) a physics-aware multi-round trajectory refinement process that narrows down candidate adversarial maneuvers, and (2) a tailored trajectory scoring function that encourages realistic yet adversarial behavior while maintaining compatibility with downstream video synthesis. As tested on the nuScenes dataset, Challenger generates a diverse range of aggressive driving scenarios-including cut-ins, sudden lane changes, tailgating, and blind spot intrusions-and renders them into multiview photorealistic videos. Extensive evaluations show that these scenarios significantly increase the collision rate of state-of-the-art end-to-end AD models (UniAD, VAD, SparseDrive, and DiffusionDrive), and importantly, adversarial behaviors discovered for one model often transfer to others.

Figures

Figures reproduced from arXiv: 2505.15880 by the authors.

Figure 1
Figure 1. Photorealistic adversarial driving videos generated by Challenger. Each scenario includes an adversarial vehicle—highlighted with a white 3D bounding box in camera views and depicted as a red rectangle in the bird’s-eye view (BEV) map—that is intentionally designed to challenge the ego vehicle through aggressive or unexpected maneuvers. Challenger autonomously produces these videos using a diffusion-based trajectory… view at source ↗
Figure 2
Figure 2. Challenger Overview. Challenger first ingests 3D bounding boxes and BEV road maps from a real-world dataset (e.g., nuScenes) to initialize a driving scene. It then randomly selects a background vehicle to act as the adversarial agent, while leaving all other participants unchanged. At fixed keyframes, Challenger plans the adversarial vehicle’s trajectory and executes it continu￾ously between keyframes. At each plann… view at source ↗
Figure 3
Figure 3. Failure cases of an E2E AD model in adversarial scenarios generated by [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Attack Transferability. This figure illustrates the failure rates of victim models when evaluated on ad￾versarial driving scenarios filtered based on the perfor￾mance of a surrogate model. A failure is recorded when the victim model collides with another object in at l…
Figure 5
Figure 5. Figure 5: Representative video frames from datasets. The re-rendered and adversarial datasets maintain photorealistic quality compa￾rable to the original nuScenes dataset. Round 0 Round 1 Round 2 Round 3 Round 4 w/o TS [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 7
Figure 7. Figure 7: Runtime Composition. Percentage of total time spent on each major component in the Challenger pipeline. Component Runtime (s) Traj. Sampling 17.36 Physics 211.36 Traj. Scoring 194.72 Rendering 395.18 Total 818.62 [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Video Quality Evaluation Pipeline. The video quality evaluation pipeline consists of two steps: (1) segmenting vehicles in the video frames using Grounded-SAM 2, and (2) computing the subject consistency metric and imaging quality metric from VBench. This section descr…
Figure 9
Figure 9. Figure 9: Example Scenes from the Adv-nuSc Dataset. We recommend the reader to have a look at our project page for dynamic videos. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Extending to Multiple Adversarial Vehicles. The extension process is simple and mod￾ular. After generating a scenario with one adversarial vehicle and before rendering, we can simply feed the generated scene into Challenger again, and designate another vehicle as the …
Figure 11
Figure 11. Figure 11: Example Scenes with Multiple Adversarial Vehicles. We recommend the reader to have a look at our project page for dynamic videos. The final scene, with all adversarial vehicles included, is then rendered as usual. While each adver￾sarial vehicle is optimized independe…
Figure 12
Figure 12. Figure 12: Misforecasting failure cases of UniAD [8]. (a) (b) (c) (d) [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Misforecasting failure cases of SparseDrive [ [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]
Figure 14
Figure 14. Figure 14: Misplanning failure cases of UniAD [8]. (a) (b) (c) (d) [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: Misplanning failure cases of SparseDrive [ [PITH_FULL_IMAGE:figures/full_fig_p023_15.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. muSync-GS: Physics-Synchronized Driving Video Synthesis for Weather and Geometric Road Hazards

    cs.CV 2026-08 conditional novelty 6.0 of 10

    muSync-GS couples weather and road-shape edits in driving videos to a calibrated vehicle-dynamics model, so the synthesized ego motion and telemetry change with the same controls that drive the visual edits.

  2. Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Models

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A new 80K-clip dataset of unstructured driving scenarios with Q&A annotations improves VLA performance on NeuroNCAP and nuScenes benchmarks.

  3. CRUISE: Cooperative Reconstruction and Editing in V2X Scenarios using Gaussian Splatting

    cs.CV 2025-07 conditional novelty 5.0 of 10

    CRUISE reconstructs real V2X driving scenes as editable Gaussians, then shows that training on its generated data improves 3D detection and tracking on the V2X-Seq benchmark.

  4. DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Driving

    cs.CV 2025-07 unverdicted novelty 5.0 of 10

    DIVER uses RL-guided diffusion to produce diverse feasible trajectories from one ground-truth path, addressing mode collapse in imitation learning for autonomous driving.

  5. Interactive Adversarial Testing of Autonomous Vehicles with Adjustable Confrontation Intensity

    cs.RO 2025-07 conditional novelty 4.0 of 10

    ExamPPO trains an adversarial surrounding vehicle with a confrontation-intensity dial and attention-based policy, producing graded, scenario-adaptive failures in simulated AV policies.

Reference graph

Works this paper leans on

69 extracted references · 39 canonical work pages · cited by 5 Pith papers

  1. [1]

    Caesar, V

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom. nuscenes: A multimodal dataset for autonomous driving. In CVPR, 2020

    work page 2020

  2. [2]

    Contributors

    O. Contributors. Openscene: The largest up-to-date 3d occupancy prediction benchmark in autonomous driving. https://github.com/OpenDriveLab/OpenScene, 2023

    work page 2023

  3. [3]

    Dauner, M

    D. Dauner, M. Hallgarten, T. Li, X. Weng, Z. Huang, Z. Yang, H. Li, I. Gilitschenski, B. Ivanovic, M. Pavone, A. Geiger, and K. Chitta. Navsim: Data-driven non-reactive au- tonomous vehicle simulation and benchmarking. Advances in Neural Information Processing Systems, 37:28706–28719, Dec. 2024

    work page 2024

  4. [4]

    X. Jia, Z. Yang, Q. Li, Z. Zhang, and J. Yan. Bench2drive: Towards multi-ability benchmarking of closed-loop end-to-end autonomous driving. Nov. 2024. URL https://openreview. net/forum?id=y09S5rdaWY#discussion

    work page 2024

  5. [5]

    S. Feng, X. Yan, H. Sun, Y . Feng, and H. X. Liu. Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment. Nature communications, 12(1):748, 2021

    work page 2021

  6. [6]

    W. Ding, B. Chen, M. Xu, and D. Zhao. Learning to collide: An adaptive safety-critical scenar- ios generating method. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), page 2243–2250, Oct. 2020. doi:10.1109/IROS45743.2020.9340696. URL https://ieeexplore.ieee.org/abstract/document/9340696

    arXiv 2020

  7. [7]

    Ljungbergh, A

    W. Ljungbergh, A. Tonderski, J. Johnander, H. Caesar, K. Åström, M. Felsberg, and C. Pe- tersson. NeuroNCAP: Photorealistic Closed-Loop Safety Testing for Autonomous Driving , volume 15088 of Lecture Notes in Computer Science, page 161–177. Springer Nature Switzer- land, Cham, 2025. ISBN 978-3-031-73403-8. doi:10.1007/978-3-031-73404-5_10. URL https://lin...

    work page doi:10.1007/978-3-031-73404-5_10 2025

  8. [8]

    Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang, et al. Planning-oriented autonomous driving. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pages 17853–17862, 2023

    work page 2023

Show all 69 references
  1. [10]

    W. Sun, X. Lin, Y . Shi, C. Zhang, H. Wu, and S. Zheng. Sparsedrive: End-to-end autonomous driving via sparse scene representation. (arXiv:2405.19620), May 2024. doi:10.48550/arXiv. 2405.19620. URL http://arxiv.org/abs/2405.19620. arXiv:2405.19620

    2024 doi arXiv

  2. [11]

    B. Liao, S. Chen, H. Yin, B. Jiang, C. Wang, S. Yan, X. Zhang, X. Li, Y . Zhang, Q. Zhang, and X. Wang. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving. (arXiv:2411.15139), Nov. 2024. doi:10.48550/arXiv.2411.15139. URL http://arxiv.org/ abs/2411.151...

    2024 doi arXiv

  3. [12]

    S. Feng, H. Sun, X. Yan, H. Zhu, Z. Zou, S. Shen, and H. X. Liu. Dense reinforcement learning for safety validation of autonomous vehicles. Nature, 615(7953):620–627, 2023

    2023

  4. [13]

    X. Yan, Z. Zou, S. Feng, H. Zhu, H. Sun, and H. X. Liu. Learning naturalistic driving environ- ment with statistical realism. Nature communications, 14(1):2037, 2023

    2023

  5. [14]

    B. Yang, H. Su, N. Gkanatsios, T.-W. Ke, A. Jain, J. Schneider, and K. Fragkiadaki. Diffusion- es: Gradient-free planning with diffusion for autonomous and instruction-guided driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1...

    2024

  6. [15]

    Zheng, R

    Y . Zheng, R. Liang, K. Zheng, J. Zheng, L. Mao, J. Li, W. Gu, R. Ai, S. E. Li, X. Zhan, et al. Diffusion-based planning for autonomous driving with flexible guidance. arXiv preprint arXiv:2501.15564, 2025

    2025 arXiv

  7. [16]

    S. Tan, B. Ivanovic, X. Weng, M. Pavone, and P. Kraehenbuehl. Language conditioned traffic generation. arXiv preprint arXiv:2307.07947, 2023

    2023 arXiv

  8. [17]

    J. Xia, C. Xu, Q. Xu, Y . Wang, and S. Chen. Language-driven interactive traffic trajectory generation. Advances in Neural Information Processing Systems, 37:77831–77859, 2024

    2024

  9. [18]

    S. Li, T. Azfar, and R. Ke. Chatsumo: Large language model for automating traffic scenario generation in simulation of urban mobility. IEEE Transactions on Intelligent Vehicles, 2024

    2024

  10. [19]

    C. Pan, B. Yaman, T. Nesti, A. Mallik, A. G. Allievi, S. Velipasalar, and L. Ren. Vlp: Vision language planning for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14760–14769, 2024

    2024

  11. [20]

    Z. Yan, P. Li, Z. Fu, S. Xu, Y . Shi, X. Chen, Y . Zheng, Y . Li, T. Liu, C. Li, N. Luo, X. Gao, Y . Chen, Z. Wang, Y . Shi, P. Huang, Z. Han, J. Yuan, J. Gong, G. Zhou, H. Zhao, and H. Zhao. INT2: Interactive Trajectory Prediction at Intersections. In 2023 IEEE/CVF Interna- t...

    2023

  12. [21]

    R. Gao, K. Chen, E. Xie, L. Hong, Z. Li, D.-Y . Yeung, and Q. Xu. Magicdrive: Street view generation with diverse 3d geometry control. In ICLR, 2024

    2024

  13. [22]

    S. Gao, J. Yang, L. Chen, K. Chitta, Y . Qiu, A. Geiger, J. Zhang, and H. Li. Vista: A general- izable driving world model with high fidelity and versatile controllability. Advances in Neural Information Processing Systems, 37:91560–91596, 2025

    2025

  14. [23]

    B. Li, J. Deng, W. Zhang, Z. Liang, D. Du, X. Jin, and W. Zeng. Hierarchical temporal context learning for camera-based semantic scene completion. In European Conference on Computer Vision, pages 131–148. Springer, 2024

    2024

  15. [24]

    R. Gao, K. Chen, B. Xiao, L. Hong, Z. Li, and Q. Xu. Magicdrivedit: High-resolution long video generation for autonomous driving with adaptive control. arXiv preprint arXiv:2411.13807, 2024. 10

    2024 arXiv

  16. [25]

    B. Li, Y . Sun, Z. Liang, D. Du, Z. Zhang, X. Wang, Y . Wang, X. Jin, and W. Zeng. Bridging stereo geometry and bev representation with reliable mutual interaction for semantic scene completion. arXiv preprint arXiv:2303.13959, 2023

    2023 arXiv

  17. [26]

    Y . Wang, J. He, L. Fan, H. Li, Y . Chen, and Z. Zhang. Driving into the future: Multiview visual forecasting and planning with world model for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14749–14759, 2024

    2024

  18. [27]

    J. Mao, B. Li, B. Ivanovic, Y . Chen, Y . Wang, Y . You, C. Xiao, D. Xu, M. Pavone, and Y . Wang. Dreamdrive: Generative 4d scene modeling from street view images.arXiv preprint arXiv:2501.00601, 2024

    2024 arXiv

  19. [28]

    L. Wang, W. Zheng, D. Du, Y . Zhang, Y . Ren, H. Jiang, Z. Cui, H. Yu, J. Zhou, J. Lu, et al. Stag-1: Towards realistic 4d driving simulation with video generation model. arXiv preprint arXiv:2412.05280, 2024

    2024 arXiv

  20. [29]

    Z. Wu, T. Liu, L. Luo, Z. Zhong, J. Chen, H. Xiao, C. Hou, H. Lou, Y . Chen, R. Yang, Y . Huang, X. Ye, Z. Yan, Y . Shi, Y . Liao, and H. Zhao. MARS: An Instance-Aware, Modular and Realistic Simulator for Autonomous Driving. In L. Fang, J. Pei, G. Zhai, and R. Wang, editors, A...

    2024 doi

  21. [30]

    Jiang, Z

    Z. Jiang, Z. Zhu, P. Li, H.-a. Gao, T. Yuan, Y . Shi, H. Zhao, and H. Zhao. P-MapNet: Far- Seeing Map Generator Enhanced by Both SDMap and HDMap Priors. IEEE Robotics and Automation Letters, 9(10):8539–8546, Oct. 2024. ISSN 2377-3766. doi:10.1109/LRA.2024. 3447450. URL https:/...

    2024

  22. [31]

    Y . Wei, Z. Wang, Y . Lu, C. Xu, C. Liu, H. Zhao, S. Chen, and Y . Wang. Editable scene sim- ulation for autonomous driving via collaborative llm-agents. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15077–15087, 2024

    2024

  23. [32]

    X. Song, J. Zheng, S. Yuan, H.-a. Gao, J. Zhao, X. He, W. Gu, and H. Zhao. SA- GS: Scale-Adaptive Gaussian Splatting for Training-Free Anti-Aliasing, Mar. 2024. URL http://arxiv.org/abs/2403.19615. arXiv:2403.19615 [cs]

    2024 arXiv

  24. [33]

    C. Li, K. Zhou, T. Liu, Y . Wang, M. Zhuang, H.-a. Gao, B. Jin, and H. Zhao. A VD2: Accident Video Diffusion for Accident Video Description, Mar. 2025. URLhttp://arxiv.org/abs/ 2502.14801. arXiv:2502.14801 [cs]

    2025 arXiv

  25. [34]

    J. Guo, Y . Ding, X. Chen, S. Chen, B. Li, Y . Zou, X. Lyu, F. Tan, X. Qi, Z. Li, and H. Zhao. DiST-4D: Disentangled Spatiotemporal Diffusion with Metric Depth for 4D Driving Scene Generation, Mar. 2025. URL http://arxiv.org/abs/2503.15208. arXiv:2503.15208 [cs]

    2025 arXiv

  26. [35]

    X. Wang, Z. Zhu, G. Huang, X. Chen, and J. Lu. Drivedreamer: Towards real-world-driven world models for autonomous driving. ECCV, 2024

    2024

  27. [36]

    G. Zhao, X. Wang, Z. Zhu, X. Chen, G. Huang, X. Bao, and X. Wang. Drivedreamer-2: Llm- enhanced world models for diverse driving video generation.arXiv preprint arXiv:2403.06845, 2024

    2024 arXiv

  28. [37]

    R. Gao, K. Chen, Z. Li, L. Hong, Z. Li, and Q. Xu. Magicdrive3d: Controllable 3d generation for any-view rendering in street scenes. arXiv preprint arXiv:2405.14475, 2024

    2024 arXiv

  29. [38]

    B. Li, J. Guo, H. Liu, Y . Zou, Y . Ding, X. Chen, H. Zhu, F. Tan, C. Zhang, T. Wang, et al. Uniscene: Unified occupancy-centric driving scene generation. arXiv preprint arXiv:2412.05435, 2024. 11

    2024 arXiv

  30. [39]

    Y . Yan, Z. Xu, H. Lin, H. Jin, H. Guo, Y . Wang, K. Zhan, X. Lang, H. Bao, X. Zhou, et al. Streetcrafter: Street view synthesis with controllable video diffusion models. arXiv preprint arXiv:2412.13188, 2024

    2024 arXiv

  31. [40]

    Q. Wang, L. Fan, Y . Wang, Y . Chen, and Z. Zhang. Freevs: Generative view synthesis on free driving trajectory. arXiv preprint arXiv:2410.18079, 2024

    2024 arXiv

  32. [41]

    Bojarski, D

    M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang, et al. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316, 2016

    2016 arXiv

  33. [42]

    Michelmore, M

    R. Michelmore, M. Kwiatkowska, and Y . Gal. Evaluating uncertainty quantification in end-to- end autonomous driving control. arXiv preprint arXiv:1811.06817, 2018

    2018 arXiv

  34. [43]

    Zheng, R

    W. Zheng, R. Song, X. Guo, and L. Chen. Genad: Generative end-to-end autonomous driving. arXiv preprint arXiv:2402.11502, 2024

    2024 arXiv

  35. [44]

    Chitta, A

    K. Chitta, A. Prakash, and A. Geiger. Neat: Neural attention fields for end-to-end autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15793–15803, 2021

    2021

  36. [45]

    Z. Xu, Y . Zhang, E. Xie, Z. Zhao, Y . Guo, K.-Y . K. Wong, Z. Li, and H. Zhao. Drivegpt4: Interpretable end-to-end autonomous driving via large language model. IEEE Robotics and Automation Letters, 2024

    2024

  37. [46]

    H. Yu, W. Yang, J. Zhong, Z. Yang, S. Fan, P. Luo, and Z. Nie. End-to-end autonomous driving through v2x cooperation. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 39, pages 9598–9606, 2025

    2025

  38. [47]

    S. Xing, C. Qian, Y . Wang, H. Hua, K. Tian, Y . Zhou, and Z. Tu. Openemma: Open-source multimodal model for end-to-end autonomous driving. In Proceedings of the Winter Confer- ence on Applications of Computer Vision, pages 1001–1009, 2025

    2025

  39. [48]

    Zheng, X

    Y . Zheng, X. Li, P. Li, Y . Zheng, B. Jin, C. Zhong, X. Long, H. Zhao, and Q. Zhang. MonoOcc: Digging into Monocular Semantic Occupancy Prediction. In 2024 IEEE Interna- tional Conference on Robotics and Automation (ICRA), pages 18398–18405, May 2024. doi: 10.1109/ICRA57147.2...

    2024

  40. [49]

    K. Ding, B. Chen, Y . Su, H.-a. Gao, B. Jin, C. Sima, X. Li, W. Zhang, P. Barsch, H. Li, and H. Zhao. Hint-AD: Holistically Aligned Interpretability in End-to-End Autonomous Driving. Sept. 2024. URL https://openreview.net/forum?id=KcW31O0PtL

    2024

  41. [50]

    D. Chen, B. Zhou, V . Koltun, and P. Krähenbühl. Learning by cheating, 2019. URL https: //arxiv.org/abs/1912.12294

    2019 arXiv

  42. [51]

    S. Choi, J. Kim, J. Yun, and J. W. Choi. R-pred: Two-stage motion prediction via tube- query attention-based trajectory refinement. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8525–8535, 2023

    2023

  43. [52]

    I. Bae, J. Oh, and H.-G. Jeon. Eigentrajectory: Low-rank descriptors for multi-modal trajectory forecasting. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10017–10029, 2023

    2023

  44. [53]

    Bae, Y .-J

    I. Bae, Y .-J. Park, and H.-G. Jeon. Singulartrajectory: Universal trajectory predictor using diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17890–17901, 2024. 12

    2024

  45. [54]

    Sarva, J

    J. Sarva, J. Wang, J. Tu, Y . Xiong, S. Manivasagam, and R. Urtasun. Adv3d: Generating safety-critical 3d objects through closed-loop simulation. arXiv preprint arXiv:2311.01446 , 2023

    2023 arXiv

  46. [55]

    Zhang, Z

    L. Zhang, Z. Peng, Q. Li, and B. Zhou. Cat: Closed-loop adversarial training for safe end-to- end driving. In Conference on Robot Learning, pages 2357–2372. PMLR, 2023

    2023

  47. [56]

    Abeysirigoonawardena, K

    Y . Abeysirigoonawardena, K. Xie, C. Chen, S. Hosseini, R. Chen, R. Wang, and F. Shkurti. Generating transferable adversarial simulation scenarios for self-driving via neural rendering. arXiv preprint arXiv:2309.15770, 2023

    2023 arXiv

  48. [57]

    J. Tu, S. Suo, C. Zhang, K. Wong, and R. Urtasun. Towards scalable coverage-based testing of autonomous vehicles. In Conference on Robot Learning, pages 2611–2623. PMLR, 2023

    2023

  49. [58]

    B. Kim, C. M. Kang, S. H. Lee, H. Chae, J. Kim, C. C. Chung, and J. W. Choi. Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network, 2017. URL https://arxiv.org/abs/1704.07049

    2017 arXiv

  50. [59]

    Zhang, G

    D. Zhang, G. Wang, R. Zhu, J. Zhao, X. Chen, S. Zhang, J. Gong, Q. Zhou, W. Zhang, N. Wang, et al. Sparsead: Sparse query-centric paradigm for efficient end-to-end autonomous driving. arXiv preprint arXiv:2404.06892, 2024

    2024 arXiv

  51. [60]

    Prakash, K

    A. Prakash, K. Chitta, and A. Geiger. Multi-modal fusion transformer for end-to-end au- tonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7077–7087, 2021

    2021

  52. [61]

    Boloor, K

    A. Boloor, K. Garimella, X. He, C. Gill, Y . V orobeychik, and X. Zhang. Attacking vision- based perception in end-to-end autonomous driving models. Journal of Systems Architecture, 110:101766, 2020

    2020

  53. [62]

    Huang, S

    Z. Huang, S. Sun, J. Zhao, and L. Mao. Multi-modal policy fusion for end-to-end autonomous driving. Information Fusion, 98:101834, 2023

    2023

  54. [63]

    Dhariwal and A

    P. Dhariwal and A. Nichol. Diffusion models beat gans on image synthesis. In Ad- vances in Neural Information Processing Systems , volume 34, page 8780–8794. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper/2021/hash/ 49ad23d1ec9fa4bd8d77d02681df5cfa-Abs...

    2021

  55. [64]

    J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. In Ad- vances in Neural Information Processing Systems , volume 33, page 6840–6851. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper/2020/hash/ 4c5bcfec8584af0d967f1ab10179ca4b-Abst...

    2020

  56. [65]

    J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. Oct. 2020. URL https://openreview.net/forum?id=St1giarCHLP

    2020

  57. [66]

    Caesar, J

    H. Caesar, J. Kabzan, K. S. Tan, W. K. Fong, E. Wolff, A. Lang, L. Fletcher, O. Beijbom, and S. Omari. Nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles. (arXiv:2106.11810), Feb. 2022. doi:10.48550/arXiv.2106.11810. URL http://arxiv.org/ abs/2106.11810....

    2022 doi arXiv

  58. [67]

    Polack, F

    P. Polack, F. Altché, B. d’Andréa Novel, and A. de La Fortelle. The kinematic bicycle model: A consistent model for planning feasible trajectories for autonomous vehicles? In 2017 IEEE Intelligent Vehicles Symposium (IV), pages 812–818, 2017. doi:10.1109/IVS.2017.7995816

    2017

  59. [68]

    Huang, Y

    Z. Huang, Y . He, J. Yu, F. Zhang, C. Si, Y . Jiang, Y . Zhang, T. Wu, Q. Jin, N. Chanpaisit, Y . Wang, X. Chen, L. Wang, D. Lin, Y . Qiao, and Z. Liu. Vbench: Comprehensive benchmark suite for video generative models. page 21807–21818, 2024. URL https://openaccess. thecvf.com...

    2024

  60. [69]

    T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y . Chen, F. Yan, Z. Zeng, H. Zhang, F. Li, J. Yang, H. Li, Q. Jiang, and L. Zhang. Grounded sam: Assembling open- world models for diverse visual tasks, 2024

    2024

  61. [70]

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. Rädle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V . Alwala, N. Carion, C.-Y . Wu, R. Girshick, P. Dol- lár, and C. Feichtenhofer. Sam 2: Segment anything in images and videos, 2024. URL https://arxiv.o...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.