Pith. sign in

REVIEW 3 major objections 7 minor 36 references

Decoupled pipeline synthesizes rare crash scenarios for self-driving safety

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-09 05:24 UTC pith:EOG7HMIB

load-bearing objection Modular LLM+CARLA+3DGS pipeline for driving corner-case synthesis is well-motivated but the headline success metric is likely inflated by survivorship bias on LLM outputs. the 3 major comments →

arxiv 2607.07601 v1 pith:EOG7HMIB submitted 2026-07-08 cs.RO cs.AI

CARLA-GS: Decoupling Representation, Reasoning, and Physics Simulation for Autonomous Driving Corner-Case Synthesis

Kaicong Huang , Meng Ma , Ruimin Ke This is my paper
classification cs.RO cs.AI
keywords autonomous drivingcorner-case generation3D Gaussian Splattinglarge language modelCARLA simulatorsafety-critical scenarioneural renderingvehicle dynamics
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

CARLA-GS proposes that the hardest problem in autonomous driving safety evaluation—generating rare, photorealistic crash scenarios—can be solved by splitting the task into three independent modules: a 3D Gaussian Splatting scene for visual fidelity, a multi-agent large language model for semantic reasoning about which vehicles should collide and how, and the CARLA driving simulator for enforcing physically realistic vehicle motion. The paper claims that by keeping these modules decoupled yet tightly coupled through state re-projection, the pipeline produces videos that are simultaneously photorealistic, semantically intentional, and physically feasible—something no single-module approach has achieved. The core mechanism is a Real-Sim-Real loop: real driving data is reconstructed into an editable Gaussian scene, an LLM reasons about collision zones and generates intent-level waypoints, CARLA executes those waypoints with a PID controller to ensure kinematic feasibility, and the resulting physically valid vehicle poses are re-projected back into the Gaussian scene for ego-centric rendering. The paper demonstrates this on Waymo Open Dataset sequences, synthesizing 85 corner cases across three behavior types: sudden braking, sudden cut-in, and lane drift.

Core claim

The central discovery is that the division of labor itself is the key contribution: when an LLM is restricted to high-level waypoint generation (rather than low-level control), when CARLA handles physics (rather than the LLM), and when Gaussian Splatting handles rendering (rather than either), the combined pipeline achieves a 92.5% success rate at generating safety-critical interactions with minimum time-to-collision of 0.472 seconds, while simultaneously producing smoother trajectories than LLM-only generation (lateral acceleration reduced from 2.66 to 2.02 m/s²) and photorealistic renderings with geometry-consistent surface normals. The paper also shows that geometry-consistent constraints

What carries the argument

The pipeline has three decoupled modules: (1) a Street Gaussians backbone with flattening, normal, and geometry-consistency losses for editable 3D scene reconstruction; (2) a two-agent LLM chain where Agent<Zone> performs risk analysis to select an adversarial vehicle and collision zone, and Agent<Trajectory> generates time-stamped waypoints ending at a sampled collision target point; and (3) CARLA's VehiclePIDController executing the waypoints, with simulated poses re-projected into the Gaussian scene via direct 6-DOF pose replacement of the corresponding Gaussian actor. A 3D foundation model (SAM-3D) replaces poorly-observed vehicles with pre-reconstructed assets to mitigate rendering tear

Load-bearing premise

The pipeline depends on the LLM reliably producing valid, coherent waypoint trajectories from scene descriptions—but manual inspection shows only 29.4% of LLM-generated outputs are valid, meaning the automation and scalability of the full pipeline currently rests on a fragile reasoning step.

What would settle it

If the LLM-generated waypoints consistently fail to produce the intended collision behavior after CARLA execution, or if the re-projected Gaussian renderings show visible geometric inconsistency under the novel vehicle poses, the core claim of photorealistic and physically feasible corner-case generation would not hold.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the decoupled architecture is sound, safety-critical scenario generation could scale by swapping any single module (e.g., a better physics simulator or a newer LLM) without redesigning the pipeline.
  • The 29.4% LLM validity rate suggests that the bottleneck for fully automated corner-case generation is reasoning reliability, not visual rendering or physics simulation.
  • The state re-projection bridge between CARLA and Gaussian Splatting could generalize to other sim-to-real pipelines where physics engines and neural renderers need to exchange vehicle state.
  • Closed-loop evaluation—where an autonomous driving policy reacts to the generated scenario in real time—would test whether synthesized corner cases actually expose planner failures.
  • If geometry-consistent constraints improve Gaussian surface quality without sacrificing photometric metrics, they could become standard regularization for sparse-viewpoint driving reconstruction.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The 70.6% LLM failure rate implies the current pipeline requires human filtering of generated scenarios, limiting scalability claims; a validation layer between the LLM and CARLA could reject infeasible waypoints before simulation.
  • The PID tracking gap between reference and executed trajectories could compound in dense traffic scenarios with multiple adversarial agents, where small pose errors change collision outcomes.
  • The three-camera reconstruction limit and 2-hour training cost per 100-frame scene suggest the pipeline is currently suited for targeted scenario synthesis rather than fleet-scale safety testing.
  • If LLM reasoning quality improves independently (as it tends to), the pipeline's success rate would improve without architectural changes, making the decoupled design a bet on modular improvement outpacing end-to-end approaches.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes CARLA-GS, a modular pipeline for synthesizing safety-critical autonomous driving corner cases. The framework decouples three components: (1) geometry-consistent 3D Gaussian Splatting for photorealistic scene reconstruction, (2) a multi-agent LLM for semantic scene reasoning and intent-level waypoint trajectory generation, and (3) CARLA-based physics simulation with PID control for kinematically feasible motion execution. Vehicle states from CARLA are re-projected into the Gaussian scene for ego-centric rendering. Experiments on the Waymo Open Dataset evaluate rendering quality (Table I), corner-case controllability (Table II), and trajectory feasibility (Table III), alongside qualitative results across three corner-case types.

Significance. The modular decoupling of representation, reasoning, and physics is a reasonable architectural choice that addresses a genuine gap: prior work treats these components in isolation. The geometry-consistent constraints on 3DGS (Eqs. 1–2) and the Real-Sim-Real bridge between CARLA and Gaussian rendering are practical contributions. The paper provides falsifiable quantitative metrics (Zone Hit, Success, MinTTC, comfort violations) and an honest computational cost analysis. However, the significance is tempered by the limited evaluation scope (8 sequences, 85 scenarios, no comparison with state-of-the-art methods) and a critical ambiguity in how the LLM's 29.4% output validity rate relates to the reported 92.5% success rate.

major comments (3)
  1. §IV.B, Table II: The paper states 'Manual inspection shows that 29.4% of LLM-generated outputs are valid' but does not clarify whether the Success rate (0.925) and Zone Hit rate (0.438) in Table II are computed over all LLM generation attempts or only over the 29.4% subset deemed valid. If metrics are computed only on valid outputs, the unconditional success rate would be approximately 0.294 × 0.925 ≈ 27%, which is below the rule-based baseline (0.613). This ambiguity is load-bearing because Table II is the primary quantitative evidence that the LLM module outperforms baselines, and the central claim of 'controllable corner-case generation' depends on this comparison. The authors must clarify the denominator and, if selection is applied, report unconditional metrics.
  2. §IV.B, Table II: The baselines (rule-based and random) are described as naive policies without semantic reasoning. The paper acknowledges that 'direct comparison with other state-of-the-art methods is difficult' due to task design differences, but the absence of any comparison with recent LLM-based or learning-based corner-case generation methods (e.g., [15], [16]) weakens the contribution claim. At minimum, the paper should discuss why these methods cannot be adapted for comparison, or provide a more competitive baseline than stochastic perturbation.
  3. §IV.A: The evaluation uses only 8 Waymo sequences to synthesize 85 corner-case scenarios. This sample size is small for claims about scalability and diversity. The paper should either expand the evaluation or explicitly acknowledge this as a limitation, particularly given that the abstract and conclusion claim the framework is 'scalable.'
minor comments (7)
  1. §IV.A: The paper references 'ChatGPT-5.2 API' which does not correspond to any known released model as of the manuscript date. The authors should verify and correct this to the actual model/API version used.
  2. Table I: The PSNR degradation from 29.703 (baseline) to 29.027 (full model) is noted as 'negligible' but represents a 0.7 dB drop. The paper should briefly discuss whether this tradeoff is acceptable for downstream tasks or whether it indicates over-regularization.
  3. Eq. (11): The loss weight λ_d and the candidate rotation set Θ are mentioned but the specific value of λ_d used in experiments is not reported. Please add this to the experimental setup.
  4. §III.B: The collision zone dimensions are parameterized by ego vehicle length L_e and width W_e, but the specific multipliers or offsets used to define zone bounds are not specified. Please clarify.
  5. Fig. 6: The trajectory visualizations lack axis labels and scale bars, making it difficult to assess spatial relationships between the ego vehicle, adversarial vehicle, and collision target point.
  6. §V: The computational cost analysis mentions 3DGS reconstruction takes ~2 hours for a 100-frame, 3-camera scene, but does not report total end-to-end time for a single corner-case synthesis cycle. A summary table would help readers assess practical deployability.
  7. The paper would benefit from a figure showing the re-projection error between CARLA-simulated trajectories and the final Gaussian-rendered vehicle positions, to validate the Real-Sim-Real bridge quantitatively rather than only qualitatively.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive review. The referee raises three major comments: (1) ambiguity about whether Table II metrics are computed over all LLM attempts or only the 29.4% valid subset, (2) absence of comparison with recent LLM-based or learning-based corner-case generation methods, and (3) limited evaluation scale (8 sequences, 85 scenarios) relative to scalability claims. We address each below. Comment 1 reveals a genuine ambiguity in the manuscript that we will fix; Comment 2 is partially addressable through expanded discussion; Comment 3 is correct and requires both a revision of claims and an explicit limitation acknowledgment.

read point-by-point responses

  1. Referee: §IV.B, Table II: Ambiguity about whether Success rate (0.925) and Zone Hit rate (0.438) are computed over all LLM generation attempts or only over the 29.4% valid subset. If only over valid outputs, the unconditional success rate would be ~27%, below the rule-based baseline (0.613).

    Authors: The referee has identified a genuine ambiguity in our manuscript, and we appreciate the careful reading. To clarify: the 29.4% validity rate and the Table II metrics measure different things and are computed over different denominators. The 29.4% figure refers to the proportion of LLM-generated waypoint sequences that satisfy all structural constraints (e.g., correct waypoint format, temporal ordering, final waypoint landing in the target zone, no premature collision before the target frame) upon manual inspection of raw LLM outputs. The Table II metrics (Zone Hit, Success, MinTTC) are computed over the full set of 85 scenarios after the CARLA PID controller has executed the trajectories — including those where the LLM output was structurally imperfect but still trackable. The PID controller can partially correct for LLM imprecision, which is why the post-execution Success rate (0.925) is higher than the raw LLM validity rate (0.294). However, we acknowledge that the manuscript does not make this denominator distinction clear, and the relationship between the two numbers is confusing as written. We will revise §IV.B to explicitly state: (a) that the 29.4% validity rate is computed over raw LLM outputs before CARLA execution, (b) that Table II metrics are computed over all 85 scenarios post-CARLA execution, and (c) that the gap between these numbers reflects the corrective role of the physics-based controller. We will also add an unconditional metric column reporting the product of validity and post-execution success to give readers the full picture. This is a case where the underlying results are sound but the presentation was genuinely ambiguous, and we will fix it. revision: yes

  2. Referee: §IV.B, Table II: Absence of comparison with recent LLM-based or learning-based corner-case generation methods (e.g., [15], [16]). The paper should discuss why these methods cannot be adapted or provide a more competitive baseline.

    Authors: We agree that the absence of direct comparison with [15] (Lu et al.) and [16] (Mei et al.) is a weakness, and we will strengthen the discussion. The core difficulty is a task-design mismatch: [15] generates corner cases as augmented trajectory data for downstream training, without requiring photorealistic in-scene rendering or physics-based execution; [16] performs online safety-critical scenario generation using retrieval-augmented LLMs but operates in a simulation-native environment (e.g., CARLA assets) rather than reconstructing from real driving data with Gaussian Splatting. Neither method produces the photorealistic, ego-centric rendered observations that are the output of our pipeline, making direct metric-level comparison infeasible without substantial reimplementation and adaptation. That said, we agree the paper should explain this more explicitly rather than simply stating 'direct comparison is difficult.' In the revision, we will: (1) add a dedicated paragraph in §IV.B explaining the specific task-design differences that prevent apples-to-apples comparison with [15] and [16], (2) discuss what aspects of those methods could in principle be adapted as baselines (e.g., using their trajectory generation modules as drop-in replacements for our LLM agent), and (3) acknowledge that a full adapted-baseline comparison is a natural next step for future work. We will also soften the contribution claim from 'outperforms baselines' to 'outperforms the tested rule-based and random baselines' to avoid overstating the comparison scope. We note honestly that we cannot add a full reimplementation of [15] or [16] within the revision cycle, so this remains a partial fix. revision: partial

  3. Referee: §IV.A: Evaluation uses only 8 Waymo sequences and 85 scenarios, which is small for scalability claims. The paper should expand evaluation or acknowledge this as a limitation, especially given 'scalable' claims in abstract and conclusion.

    Authors: The referee is correct that 8 sequences and 85 scenarios is a limited evaluation scope, and we agree that the word 'scalable' in the abstract and conclusion overstates what the current evaluation supports. We will make two changes. First, we will add an explicit limitation paragraph in §IV.A (and mirror it in the conclusion) acknowledging that the current evaluation covers 8 sequences and 85 scenarios, and that claims about scalability and diversity are not fully validated at this scale. Second, we will revise the abstract and conclusion to replace 'scalable' with more precise language — e.g., 'designed for scalability' or 'modular design that supports scalability' — making clear that scalability is an architectural property of the pipeline (each module can independently scale) rather than an empirically validated claim. We cannot expand the evaluation to substantially more sequences within the revision timeline because each sequence requires full 3DGS reconstruction (~2 hours per scene), manual scenario curation, and multi-stage pipeline execution. We are transparent about this constraint and believe the honest path is to scope the claims accordingly rather than overstate them. revision: yes

Circularity Check

0 steps flagged

No significant circularity: the pipeline integrates external tools (CARLA, ChatGPT, Street Gaussians) and evaluates against external benchmarks (Waymo Open Dataset) with standard metrics; no derivation reduces to its inputs by construction.

full rationale

The paper's central claim is that a modular pipeline (LLM + CARLA + 3DGS) can synthesize photorealistic, physically feasible corner-case scenarios. The derivation chain is: (1) geometry-consistent 3DGS reconstruction with standard flattening/normal/geometry losses (Eqs. 1-2, inspired by cited works [11,12,20,21]) — these are standard regularization terms, not circular; (2) LLM generates waypoint trajectories (Eqs. 5-7) using ChatGPT API as an external tool; (3) CARLA executes PID-controlled kinematic simulation (Eqs. 8-10) using an external simulator; (4) back-projection into Gaussian scene for rendering. Evaluation uses external data (Waymo Open Dataset), standard metrics (PSNR, SSIM, LPIPS, TTC, lateral acceleration, curvature rate), and rule-based/random baselines. The 29.4% LLM validity rate and potential survivorship bias in Table II is a correctness/selection-bias concern, not a circularity issue — the metrics are not defined in terms of the pipeline's outputs. No self-citation chain is load-bearing: reference [18] (Huang et al., 'Real2sim') shares an author but is cited only for related work on physics-augmented Gaussians, not as a premise for any derivation. The flattening constraint (Eq. 1) and normal constraint (Eq. 2) follow standard formulations from 2DGS and surface reconstruction literature. No equation reduces to its inputs by construction, and no 'prediction' is a renamed fit.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 0 invented entities

The paper does not invent new physical entities or fundamental mathematical constructs. It introduces a pipeline architecture and associated loss functions, but these are engineering contributions rather than new axioms or postulated objects.

free parameters (3)
  • Loss weights for geometric constraints = 0.1
    The three geometric consistency loss terms (L_flat, L_normal, L_geo) are each weighted by 0.1, a value chosen empirically.
  • PID controller gains = k_p=2.0, k_d=0.2, k_i=0.0
    The longitudinal and lateral PID gains for the CARLA controller are manually set to these values.
  • Collision zone dimensions = Unspecified, based on ego vehicle length L_e and width W_e
    The spatial bounds of the four collision zones are parameterized by ego vehicle dimensions but the exact multipliers or offsets are not detailed.
axioms (3)
  • domain assumption 3D Gaussian Splatting can be extended with geometry-consistent constraints to improve surface quality without sacrificing photometric fidelity.
    The paper assumes the proposed flattening and normal constraints will improve geometric stability for driving scenes, which is supported by the ablation in Table I.
  • domain assumption Large Language Models can perform scene-level reasoning to identify risky interactions and generate intent-level waypoint trajectories.
    The pipeline relies on the LLM's ability to understand spatial relationships and generate valid trajectories, an assumption challenged by the 29.4% validity rate reported in the paper.
  • domain assumption CARLA's internal kinematic model and PID control are sufficient to enforce dynamic feasibility for LLM-generated trajectories.
    The paper assumes that delegating low-level control to CARLA will regularize the LLM outputs into physically feasible motion, which is supported by the metrics in Table III.

pith-pipeline@v1.1.0-glm · 14402 in / 2306 out tokens · 437488 ms · 2026-07-09T05:24:34.560064+00:00 · methodology

0 comments
read the original abstract

Safety evaluation for autonomous driving is dominated by rare, safety-critical interactions, motivating simulators that can deliberately synthesize corner cases with photorealistic observations. Corner-case generation is inherently a multi-source problem spanning visual representation, scene reasoning, and vehicle trajectory generation and control. Prior knowledge- and model-based approaches typically focus on scene or trajectory components in isolation, while diffusion-based methods attempt end-to-end generation but still struggle to ensure spatiotemporal consistency and physical realism. To unify these aspects within a single framework, we propose CARLA-GS, a modular corner-case synthesis pipeline that decouples visual representation, semantic reasoning, and physics-based execution while maintaining tight cross-module coupling. Starting from real driving data, we reconstruct an editable gaussian scene with additional geometry-consistent constraints. A multi-agent LLM then performs scene-level reasoning to identify risky interactions and generate intent-level waypoint trajectories, while the low-level motion control is delegated to CARLA, where a PID controller ensures kinematic and dynamic feasibility. The simulated vehicle states are finally re-projected into the gaussian scene for ego-centric rendering. This design enables high-level semantic reasoning, low-level physically executable motion, and photorealistic corner-case generation within a unified pipeline. Experiments on the Waymo Open Dataset show, both quantitatively and qualitatively, that our framework enables controllable corner-case generation and produces photorealistic, spatiotemporally consistent videos aligned with semantic intent and physically feasible motion.

Figures

Figures reproduced from arXiv: 2607.07601 by Kaicong Huang, Meng Ma, Ruimin Ke.

Figure 1
Figure 1. Figure 1: Overall framework of CARLA-GS. We decouple scene representation, risk reasoning, and physics simulation to leverage the strengths of each [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Construction of the supervision mask for geometry-consistent [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Vehicle replacement pipeline. We first select frames where the [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the LLM-driven corner-case generation pipeline. The [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Surface normal renderings. Our geometry-consistency constraints produce finer surface details and fewer artifacts than the baseline. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: CARLA-based realization of LLM-generated trajectories. Left: PID [PITH_FULL_IMAGE:figures/full_fig_p006_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Behavior risk analysis results produced by the LLM agent. Given [PITH_FULL_IMAGE:figures/full_fig_p006_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Final Gaussian renderings of the three synthesized corner cases. Left: raw input frames. Right: edited renderings after LLM-driven reasoning and [PITH_FULL_IMAGE:figures/full_fig_p007_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 36 canonical work pages · 9 internal anchors

  1. [1]

    Curse of rarity for autonomous vehicles,

    H. X. Liu and S. Feng, “Curse of rarity for autonomous vehicles,” nature communications, vol. 15, no. 1, p. 4808, 2024

    work page 2024

  2. [2]

    Omnidrive: A holistic vision-language dataset for autonomous driving with counterfactual reasoning,

    S. Wang, Z. Yu, X. Jiang, S. Lan, M. Shi, N. Chang, J. Kautz, Y . Li, and J. M. Alvarez, “Omnidrive: A holistic vision-language dataset for autonomous driving with counterfactual reasoning,” inProceedings of the computer vision and pattern recognition conference, 2025, pp. 22 442–22 452

    work page 2025

  3. [3]

    NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)

    K. Gao, Y . Gao, H. He, D. Lu, L. Xu, and J. Li, “Nerf: Neural radiance field in 3d vision, a comprehensive review,”arXiv preprint arXiv:2210.00379, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  4. [4]

    Drive- dreamer: Towards real-world-drive world models for autonomous driving,

    X. Wang, Z. Zhu, G. Huang, X. Chen, J. Zhu, and J. Lu, “Drive- dreamer: Towards real-world-drive world models for autonomous driving,” inEuropean conference on computer vision. Springer, 2024, pp. 55–72

    work page 2024

  5. [5]

    GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving

    L. Russell, A. Hu, L. Bertoni, G. Fedoseev, J. Shotton, E. Arani, and G. Corrado, “Gaia-2: A controllable multi-view generative world model for autonomous driving,”arXiv preprint arXiv:2503.20523, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  6. [6]

    3d gaussian splatting for real-time radiance field rendering

    B. Kerbl, G. Kopanas, T. Leimk ¨uhler, G. Drettakiset al., “3d gaussian splatting for real-time radiance field rendering.”ACM Trans. Graph., vol. 42, no. 4, pp. 139–1, 2023

    work page 2023

  7. [7]

    4d gaussian splatting for real-time dynamic scene rendering,

    G. Wu, T. Yi, J. Fang, L. Xie, X. Zhang, W. Wei, W. Liu, Q. Tian, and X. Wang, “4d gaussian splatting for real-time dynamic scene rendering,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 20 310–20 320

    work page 2024

  8. [8]

    Drivinggaussian: Composite gaussian splatting for surrounding dy- namic autonomous driving scenes,

    X. Zhou, Z. Lin, X. Shan, Y . Wang, D. Sun, and M.-H. Yang, “Drivinggaussian: Composite gaussian splatting for surrounding dy- namic autonomous driving scenes,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 21 634–21 643

    work page 2024

  9. [9]

    Street gaussians: Modeling dynamic urban scenes with gaussian splatting,

    Y . Yan, H. Lin, C. Zhou, W. Wang, H. Sun, K. Zhan, X. Lang, X. Zhou, and S. Peng, “Street gaussians: Modeling dynamic urban scenes with gaussian splatting,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 156–173

    work page 2024

  10. [10]

    Cruise: Cooperative reconstruction and editing in v2x scenarios using gaussian splatting,

    H. Xu, S. Zhang, P. Li, B. Ye, X. Chen, H.-a. Gao, J. Zheng, X. Song, Z. Peng, R. Miaoet al., “Cruise: Cooperative reconstruction and editing in v2x scenarios using gaussian splatting,” in2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2025, pp. 12 518–12 525

    work page 2025

  11. [11]

    2d gaussian splat- ting for geometrically accurate radiance fields,

    B. Huang, Z. Yu, A. Chen, A. Geiger, and S. Gao, “2d gaussian splat- ting for geometrically accurate radiance fields,” inACM SIGGRAPH 2024 conference papers, 2024, pp. 1–11

    work page 2024

  12. [12]

    Pgsr: Planar-based gaussian splatting for efficient and high-fidelity surface reconstruction,

    D. Chen, H. Li, W. Ye, Y . Wang, W. Xie, S. Zhai, N. Wang, H. Liu, H. Bao, and G. Zhang, “Pgsr: Planar-based gaussian splatting for efficient and high-fidelity surface reconstruction,”IEEE Transactions on Visualization and Computer Graphics, vol. 31, no. 9, pp. 6100– 6111, 2024

    work page 2024

  13. [13]

    Drive- gen: Towards infinite diverse traffic scenarios with large models,

    S. Zhang, J. Tian, Z. Zhu, S. Huang, J. Yang, and W. Zhang, “Drive- gen: Towards infinite diverse traffic scenarios with large models,” in 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2025, pp. 10 100–10 107

    work page 2025

  14. [14]

    CC-SGG: Corner Case Scenario Generation using Learned Scene Graphs

    G. Drayson, E. Panagiotaki, D. Omeiza, and L. Kunze, “Cc-sgg: Corner case scenario generation using learned scene graphs,”arXiv preprint arXiv:2309.09844, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  15. [15]

    Realistic Corner Case Generation for Autonomous Vehicles with Multimodal Large Language Model

    Q. Lu, M. Ma, X. Dai, X. Wang, and S. Feng, “Realistic corner case generation for autonomous vehicles with multimodal large language model,”arXiv preprint arXiv:2412.00243, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  16. [16]

    Seeking to Collide: Online Safety-Critical Scenario Generation for Autonomous Driving with Retrieval Augmented Large Language Models

    Y . Mei, T. Nie, J. Sun, and Y . Tian, “Seeking to collide: Online safety- critical scenario generation for autonomous driving with retrieval augmented large language models,”arXiv preprint arXiv:2505.00972, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  17. [17]

    Phys- gaussian: Physics-integrated 3d gaussians for generative dynamics,

    T. Xie, Z. Zong, Y . Qiu, X. Li, Y . Feng, Y . Yang, and C. Jiang, “Phys- gaussian: Physics-integrated 3d gaussians for generative dynamics,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 4389–4398

    work page 2024

  18. [18]

    Real2Sim: A Physics-driven and Editable Gaussian Splatting Framework for Autonomous Driving Scenes

    K. Huang, T. Azfar, W. Shi, and R. Ke, “Real2sim: A physics-driven and editable gaussian splatting framework for autonomous driving scenes,”arXiv preprint arXiv:2605.13591, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  19. [19]

    Vr-gs: A physical dynamics-aware interactive gaussian splatting system in virtual reality,

    Y . Jiang, C. Yu, T. Xie, X. Li, Y . Feng, H. Wang, M. Li, H. Lau, F. Gao, Y . Yanget al., “Vr-gs: A physical dynamics-aware interactive gaussian splatting system in virtual reality,” inACM SIGGRAPH 2024 conference papers, 2024, pp. 1–1

    work page 2024

  20. [20]

    Vid2sim: Realistic and interactive simulation from video for urban navigation,

    Z. Xie, Z. Liu, Z. Peng, W. Wu, and B. Zhou, “Vid2sim: Realistic and interactive simulation from video for urban navigation,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 1581–1591

    work page 2025

  21. [21]

    Vr- robo: A real-to-sim-to-real framework for visual robot navigation and locomotion,

    S. Zhu, L. Mou, D. Li, B. Ye, R. Huang, and H. Zhao, “Vr- robo: A real-to-sim-to-real framework for visual robot navigation and locomotion,”IEEE Robotics and Automation Letters, 2025

    work page 2025

  22. [22]

    Carla: An open urban driving simulator,

    A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, “Carla: An open urban driving simulator,” inConference on robot learning. PMLR, 2017, pp. 1–16

    work page 2017

  23. [23]

    STSBench: A Spatio-temporal Scenario Benchmark for Multi-modal Large Language Models in Autonomous Driving

    C. Fruhwirth-Reisinger, D. Mali ´c, W. Lin, D. Schinagl, S. Schulter, and H. Possegger, “Stsbench: A spatio-temporal scenario benchmark for multi-modal large language models in autonomous driving,”arXiv preprint arXiv:2506.06218, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  24. [24]

    Nerf: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoor- thi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,”Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2021

    work page 2021

  25. [25]

    Mine: Towards continuous depth mpi with nerf for novel view synthesis,

    J. Li, Z. Feng, Q. She, H. Ding, C. Wang, and G. H. Lee, “Mine: Towards continuous depth mpi with nerf for novel view synthesis,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 12 578–12 588

    work page 2021

  26. [26]

    Mapnerf: Incorporating map priors into neural radiance fields for driving view simulation,

    C. Wu, J. Sun, Z. Shen, and L. Zhang, “Mapnerf: Incorporating map priors into neural radiance fields for driving view simulation,” in2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 7082–7088

    work page 2023

  27. [27]

    Lane- segnet: Map learning with lane segment perception for autonomous driving,

    T. Li, P. Jia, B. Wang, L. Chen, K. Jiang, J. Yan, and H. Li, “Lane- segnet: Map learning with lane segment perception for autonomous driving,”arXiv preprint arXiv:2312.16108, 2023

    work page arXiv 2023

  28. [28]

    3D Gaussian Splatting in Robotics: A Survey

    S. Zhu, G. Wang, X. Kong, D. Kong, and H. Wang, “3d gaussian splatting in robotics: A survey,”arXiv preprint arXiv:2410.12262, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  29. [29]

    Desire-gs: 4d street gaussians for static-dynamic decomposition and surface reconstruction for urban driving scenes,

    C. Peng, C. Zhang, Y . Wang, C. Xu, Y . Xie, W. Zheng, K. Keutzer, M. Tomizuka, and W. Zhan, “Desire-gs: 4d street gaussians for static-dynamic decomposition and surface reconstruction for urban driving scenes,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 6782–6791

    work page 2025

  30. [30]

    Lodge: Level- of-detail large-scale gaussian splatting with efficient rendering,

    J. Kulhanek, M.-J. Rakotosaona, F. Manhardt, C. Tsalicoglou, M. Niemeyer, T. Sattler, S. Peng, and F. Tombari, “Lodge: Level- of-detail large-scale gaussian splatting with efficient rendering,”arXiv preprint arXiv:2505.23158, 2025

    work page arXiv 2025

  31. [31]

    Rp- slam: Real-time photorealistic slam with efficient 3d gaussian splat- ting,

    L. Bai, C. Tian, J. Yang, S. Zhang, M. Suganuma, and T. Okatani, “Rp- slam: Real-time photorealistic slam with efficient 3d gaussian splat- ting,”IEEE Transactions on Visualization and Computer Graphics, 2025

    work page 2025

  32. [32]

    Deepaccident: A motion and accident prediction benchmark for v2x autonomous driving,

    T. Wang, S. Kim, J. Wenxuan, E. Xie, C. Ge, J. Chen, Z. Li, and P. Luo, “Deepaccident: A motion and accident prediction benchmark for v2x autonomous driving,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 6, 2024, pp. 5599–5606

    work page 2024

  33. [33]

    Terasim- world: Worldwide safety-critical data synthesis for end-to-end au- tonomous driving,

    J. Wang, H. Sun, X. Yan, S. Feng, J. Gao, and H. X. Liu, “Terasim- world: Worldwide safety-critical data synthesis for end-to-end au- tonomous driving,”arXiv preprint arXiv:2509.13164, 2025

    work page arXiv 2025

  34. [34]

    Scalability in perception for autonomous driving: Waymo open dataset,

    P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V . Patnaik, P. Tsui, J. Guo, Y . Zhou, Y . Chai, B. Caineet al., “Scalability in perception for autonomous driving: Waymo open dataset,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 2446–2454

    work page 2020

  35. [35]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhouet al., “Chain-of-thought prompting elicits reasoning in large language models,”Advances in neural information processing systems, vol. 35, pp. 24 824–24 837, 2022

    work page 2022

  36. [36]

    SAM 3D: 3Dfy Anything in Images

    X. Chen, F.-J. Chu, P. Gleize, K. J. Liang, A. Sax, H. Tang, W. Wang, M. Guo, T. Hardin, X. Liet al., “Sam 3d: 3dfy anything in images,” arXiv preprint arXiv:2511.16624, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025