Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Occ-LLM claims a large language model reading motion-separated 3D occupancy beats specialized driving models on forecasting, planning, and scene QA.

desk verdict First occupancy-based LLM for driving with big claimed gains, but the core separation equations are inverted in the write-up and the support is otherwise shaky; worth reviewing after major fixes. read the letter →

arxiv 2502.06419 v1 pith:VLKXXIN2 submitted 2025-02-10 cs.RO

Tianshuo Xu , Hao Lu , Xu Yan , Yingjie Cai , Bingbing Liu , Yingcong Chen This is my paper · ORCID
classification cs.RO
keywords occupancyrepresentationlargelanguagemodelsautonomousdriving4Dforecastingself-egoplanningscenequestionansweringvariationalautoencodermotionseparation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces Occ-LLM, a system that turns 3D occupancy grids (the standard representation of which voxels in a driving scene are filled and by what) into tokens a large language model can process. Its core claim is that separating moving objects from static scenery before compressing the occupancy grid with a variational autoencoder lets the model forecast future occupancy and plan the ego vehicle's trajectory better than specialized single-task methods. The authors report a 3-second-average occupancy IoU of 32.52% versus 26.63% for the prior best method, and a planning L2 error of 0.28 meters versus 1.17 meters. The same model also answers questions about the scene, suggesting occupancy can serve as a unified input modality for prediction, planning, and understanding.

What carries the argument

The load-bearing object is the Motion Separation Variational Autoencoder (MS-VAE), a VQ-VAE variant with two codebooks (one for movable voxels, one for static voxels) sharing a single encoder and decoder, plus a lightweight 3D convolution before the encoder and after the decoder to preserve volumetric structure. MS-VAE turns a 200-by-200-by-16 occupancy grid into a compact token sequence that the LLM can ingest, and the motion/static split shifts the model's capacity toward dynamic trajectories while still reconstructing static scenes. Two supporting mechanisms are the patchify step (patch size 10) that flattens latent grids into tokens, and the use of <occ> and </occ> tokens at frame boundaries to prevent positional drift between consecutive occupancy frames.

What would settle it

Inspect Section III-A's Eqs. (5)-(6) against the released MS-VAE code to verify that movable voxels are actually kept in the motion stream as the prose describes, then run the NuScenes validation set with the motion-separation module ablated and compare the 3-second IoU against the reported 32.52% versus the 20.67% baseline.

Watch

Extended reading notes

Core claim

The central discovery is that occupancy, when tokenized appropriately, can serve as a native modality for a large language model in autonomous driving. The proposed Motion Separation Variational Autoencoder (MS-VAE) splits each occupancy frame into movable and static voxels, encodes the two streams with a shared encoder but separate codebooks, and reconstructs the raw grid by merging the decoded streams. On the NuScenes benchmark, Occ-LLM reports 32.52% average IoU and 20.99% average mIoU for 4D occupancy forecasting over three seconds, outperforming the strongest prior occupancy world model (26.63% IoU, 17.14% mIoU), and reduces the 3-second average L2 planning error to 0.28 meters from 1.17 meters. The same model performs occupancy-based scene question answering, indicating the LLM can use this representation for scene understanding as well as prediction and planning.

Load-bearing premise

The whole pipeline assumes the movable-class indicator that drives the motion/static separation receives accurate semantic labels from the upstream camera-based occupancy predictor; if those labels are noisy or wrong, the separation corrupts the representation and the reported gains would likely not hold.

Editorial extensions

If this is right

  • Occupancy can act as a universal input modality for an LLM-based autonomous driving system, supporting forecasting, planning, and question answering in one model.
  • Adding pre-fusion of ego-action latents with occupancy lowers planning L2 error from 0.82 meters to 0.69 meters, and motion separation further reduces it to 0.28 meters.
  • The MS-VAE reconstructs occupancy better than the prior VAE (IoU 62.74% versus 59.07%, mIoU 71.08% versus 60.50%), supporting the claim that separating movable and static voxels improves compression quality.
  • Piping predicted occupancy from BevFormer, BevDet, or FBOCC through Occ-LLM consistently improves forecasting IoU and planning L2 compared with the same predictors paired with OccWorld.
  • A patch size of 10 balances reconstruction detail and token length, indicating that the tokenization scheme, not just the LLM, is a key contributor to the reported performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the motion/static split is a general tokenization recipe: any representation with a semantic axis that strongly affects the downstream task could be separated into two codebooks before being fed to a transformer.
  • A natural extension is to replace the fixed movable-class set with a learned or open-vocabulary grouping, which would let the model adapt to rare or unknown object categories rather than relying on hand-defined classes.
  • If the reported planning gains transfer to closed-loop evaluation, occupancy-tokenized LLMs could serve as a single interpretable controller that explains its own decisions in natural language.
  • The same two-codebook separation idea could be applied to other 3D scene representations, such as signed distance fields or neural radiance fields, wherever foreground and background have very different dynamics.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Occ-LLM, an LLM-based framework for autonomous driving that consumes 3D occupancy representations. A Motion Separation Variational Autoencoder (MS-VAE) is introduced to split occupancy into moving and static components before tokenization; the tokens are fed into Llama-2 for 4D occupancy forecasting, self-ego planning, and occupancy-based question answering. Experiments on NuScenes report large improvements over OccWorld and other baselines, with headline numbers of 32.52% IoU, 20.99% mIoU, and 0.28 m L2 error. The paper also includes ablations for the VAE architecture, patch size, pre-fusion, and motion separation.

Significance. The idea of linking occupancy representations with LLMs is timely and relevant for autonomous driving, and the three-task formulation (forecasting, planning, QA) is a useful step toward unified occupancy-based world models. The exploration of multiple upstream occupancy predictors (BevFormer, BevDet, FBOCC) is also a strength, as it speaks to generality. If the reported numbers and the motion-separation mechanism were correct and reproducible, this would be a meaningful advance. However, the paper as written contains internal inconsistencies in both the core formulation and the main results table, and no code or data release is available to disambiguate them; the empirical claims are therefore not verifiable in their current form.

major comments (4)
  1. [III-A, Eqs. (5)-(7)] The equations implement the opposite of the motion/static separation described in the prose. With x_m denoting moving voxels and 1_M(x)=1 for movable classes, Eq. (5) gives x'_m = (1 - 1_M(x)) · x_m, which is zero precisely on movable voxels; Eq. (6) gives x'_s = 1_M(x) · air + (1 - 1_M(x)) · x_s, which places air in the movable positions of the static stream. Thus, if the equations are followed, the motion stream contains no moving-object content and the static stream contains air where moving objects are, so the reconstruction in Eq. (7) with mask = (x̂_m ≠ 0) cannot recover dynamic occupancy. This is the reverse of the description in Sec. III-A and Fig. 2. The issue is load-bearing because Table V attributes the largest single gain to "+Motion Separation" (IoU 20.67→32.52, mIoU 16.63→20.99, L2 0.82→0.28). Please correct the equations or the description, and confirm that the ablation and main results correspond to the corrected formulation.
  2. [Table I, OccWorld-S row] The OccWorld-S row reports IoU values 21.09, 16.17, 4.95 with an average of 5.00, but the arithmetic mean is 14.07. Moreover, the sequence 21.09, 16.17 appears to be shifted from the BevFormer+Ours row (25.35, 21.09, 16.17), suggesting a copying error. The mIoU row (0.28, 0.26, 0.24, average 0.26) is arithmetically consistent but implausibly low relative to the IoU values for the same row and relative to all other rows in the table. Please provide corrected values and clarify whether these are the official OccWorld-S results.
  3. [IV-A and Tables I, V] The evaluation protocol is underspecified: it is not stated whether Table I reports validation or test split; Tables IV and V explicitly report Trainset/Valset, but the main comparison table does not say which split is used. Since the abstract and introduction quote the Table I numbers, the split must be stated. In addition, no code or data release is mentioned, and no training details are given for the LLM fine-tuning (e.g., learning rate, number of tokens, sequence length, LoRA vs full fine-tuning). These details are necessary for the reader to verify the headline improvements, especially because the mask logic in Eqs. (5)-(6) cannot currently be resolved from the written description.
  4. [III-A and Table V] The separation relies on an indicator 1_M(x) of movable classes, but the paper does not state where these class labels come from when the upstream occupancy predictor is not a semantic occupancy model. The experiments use FBOCC, BevFormer, and BevDet as upstream predictors, but the paper does not analyze how errors or absence of semantic labels in these predictors affect the MS-VAE separation and downstream gains. Because Table V shows that motion separation accounts for the largest performance jump, this missing dependency analysis weakens the claim that MS-VAE is robust across occupancy prediction methods.
minor comments (5)
  1. [III-B] In the pre-fusion paragraph, "modulate the occupancy tents" should presumably read "modulate the occupancy tokens" or "occupancy latents".
  2. [III-A, Eq. (8)] The reconstruction term is written as E_{qφ(z_m|x_m)} log p_θ(x_m|z'_m), but z'_m is the quantized latent; please clarify the notation, and report the value of β used in training.
  3. [Table III] The "+Motion Separation" row changes the latent shape from 50,50,32 to 50,50,64 but keeps Parameters(M) at 2.30; this is likely a typo and should be corrected.
  4. [References] Several references are duplicated in the bibliography: [20] appears twice for Tri-Perspective View, [49] appears twice for SurroundOcc, and [46] is listed twice for Scene as Occupancy. Please deduplicate.
  5. [Fig. 4] The qualitative figure caption lists "Vanilla, Ours, Labels, FBOCC+Ours, OccWorld" but the legend is not clear in the figure; please add a clear legend and specify the time horizon for each panel.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline accuracies are external benchmark measurements against OccWorld, DriveLM, and other baselines, and no load-bearing result is derived from a fitted constant or from a self-citation chain.

full rationale

Occ-LLM's central claims are empirical evaluations on nuScenes. The 4D occupancy forecasting IoU/mIoU and planning L2 numbers are measured against external baselines (OccWorld, DriveLM, VAD-Base, etc.), not computed from fitted parameters or from the paper's own definitions. The MS-VAE is trained with a reconstruction and commitment loss (Eq. 8) that does not include the downstream forecasting or planning metrics, so the downstream results are not forced by the VAE training objective. The motion/static separation uses the semantic class indicator 1_M as a prior derived from occupancy semantic labels; this is an input precondition rather than a circular derivation. The comparison rows 'BevFormer+Ours', 'BevDet+Ours', and 'FBOCC+Ours' share the same occupancy inputs as the corresponding baselines, so the comparison is not defined in terms of the paper's own outputs. There are no load-bearing self-citations: the only overlapping-author reference is a CVPR workshop paper on GPT-4V in affective computing, which is unrelated to the main derivation. The paper does select the patch size (10) using the validation set and then reports validation-set numbers in Table V; this is a model-selection concern, not a fitted input renamed as a prediction. A genuine reproducibility problem exists in Eqs. 5-6, which as written invert the described motion/static separation (Eq. 5 keeps static voxels in the 'motion' stream and Eq. 6 places air at movable positions in the 'static' stream); however, this is an internal consistency/correctness issue, not circularity, because the reported numbers are not equivalent to these equations by construction. Under the stated circularity criteria, no step reduces a claimed result to its own inputs.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities or forces. It does introduce architectural components (separate codebooks, frame-delimiter tokens, pre-fusion modulation) but these are not invented entities in the sense of new particles or mediators. The central claim depends on the listed free parameters and domain assumptions, plus the correctness of the separation equations, which appear to contain a sign error.

free parameters (4)
  • patch_size = 10
    Chosen by validation reconstruction performance in Table IV; forecasting quality is sensitive to this value.
  • MS-VAE latent grid shape = 50,50,64
    Architecture choice from Table III; determines token count and reconstruction fidelity.
  • commitment_loss_weight_beta
    Appears in Eq. 8 but no value is reported; affects codebook usage and reconstruction quality.
  • pre_fusion_MLP_dimensions
    No dimensions given; action conditioning is part of the final model according to Table V.
assumptions (3)
  • domain assumption Semantic occupancy labels from upstream predictors are accurate enough for the movable-class indicator to separate moving from static cells.
    Section III-A Eq. 4 defines separation using class labels; if upstream occupancy is wrong, the separation is wrong.
  • domain assumption Llama-2 can learn 4D forecasting and planning from flattened occupancy tokens with sequence-level supervision.
    The entire framework depends on LLM sequence modeling capacity for non-linguistic tokens; no theory or small-scale evidence is provided.
  • domain assumption Reconstruction quality of MS-VAE transfers to downstream forecasting and planning.
    Ablation Table III shows reconstruction gains, but the link to downstream metrics is only correlational.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Models." pith.science (2026). https://pith.science/paper/VLKXXIN2

@misc{pith2026250206419,
  author       = {Pith},
  title        = {Pith review of: Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VLKXXIN2}},
  note         = {Machine review of arXiv:2502.06419}
}
read the original abstract

Large Language Models (LLMs) have made substantial advancements in the field of robotic and autonomous driving. This study presents the first Occupancy-based Large Language Model (Occ-LLM), which represents a pioneering effort to integrate LLMs with an important representation. To effectively encode occupancy as input for the LLM and address the category imbalances associated with occupancy, we propose Motion Separation Variational Autoencoder (MS-VAE). This innovative approach utilizes prior knowledge to distinguish dynamic objects from static scenes before inputting them into a tailored Variational Autoencoder (VAE). This separation enhances the model's capacity to concentrate on dynamic trajectories while effectively reconstructing static scenes. The efficacy of Occ-LLM has been validated across key tasks, including 4D occupancy forecasting, self-ego planning, and occupancy-based scene question answering. Comprehensive evaluations demonstrate that Occ-LLM significantly surpasses existing state-of-the-art methodologies, achieving gains of about 6\% in Intersection over Union (IoU) and 4\% in mean Intersection over Union (mIoU) for the task of 4D occupancy forecasting. These findings highlight the transformative potential of Occ-LLM in reshaping current paradigms within robotic and autonomous driving.

Figures

Figures reproduced from arXiv: 2502.06419 by the authors.

Figure 1
Figure 1. We present Occ-LLM, an occupancy-based large language model designed for autonomous driving scene prediction, planning, and understanding (zoom in for the best view). facilitates the perception of objects regardless of their spe￾cific categories, whether known or unidentified. Notably, leading automotive manufacturers, such as Tesla [43], are progressively adopting occupancy-based systems within their vehicles, high… view at source ↗
Figure 2
Figure 2. Overview of the proposed Occ-LLM framework. Initially, results from multiview cameras are converted into occupancy representations utilizing [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the positional shift problem in occupancy represen [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The evaluated methods include LiDAR-based approaches like IL [34], NMP [52], and FF [18], as well as camera￾based methods such as UniAD [19], VAD-Base [22], and OccNet [46]. We also integrate predicted occupancy data into our Occ-LLM framework, achieving higher perform…
Figure 4
Figure 4. Figure 4: Qualitative 4-D occupancy forecasting results of our Occ-LLM. “Vanilla” refers to the direct flattening of occupancy representation and its injection [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative question-answering results of our Occ-LLM. The left panel displays the raw scene data, while the right panel shows the predicted [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. COME: Adding Scene-Centric Forecasting Control to Occupancy World Model

    cs.CV 2025-06 conditional novelty 6.0 of 10

    COME adds a scene-centric forecasting branch as a ControlNet-style condition to a diffusion occupancy world model, improving static-scene consistency and beating prior methods on Occ3D-nuScenes while hiding a stronger...

Reference graph

Works this paper leans on

55 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [2]

    F. Bao, S. Nie, K. Xue, Y . Cao, C. Li, H. Su, and J. Zhu. All are worth words: A vit backbone for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 22669–22679, 2023

    work page 2023

  2. [3]

    Caesar, V

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom. nuscenes: A multimodal dataset for autonomous driving. In CVPR, pages 11621– 11631, 2020

    work page 2020

  3. [4]

    Cao and R

    A.-Q. Cao and R. de Charette. Monoscene: Monocular 3d semantic scene completion. In CVPR, pages 3991–4001, 2022

    work page 2022

  4. [5]

    L. Chen, O. Sinavski, J. H ¨unermann, A. Karnsund, A. J. Willmott, D. Birch, D. Maund, and J. Shotton. Driving with llms: Fusing object-level vector modality for explainable autonomous driving.arXiv preprint arXiv:2310.01957, 2023

    arXiv

  5. [6]

    L. Chen, P. Wu, K. Chitta, B. Jaeger, A. Geiger, and H. Li. End- to-end autonomous driving: Challenges and frontiers. arXiv preprint arXiv:2306.16927, 2023

    arXiv 2023

  6. [7]

    Chen, K.-Y

    X. Chen, K.-Y . Lin, C. Qian, G. Zeng, and H. Li. 3d sketch-aware semantic scene completion via semi-supervised structure prior. In CVPR, pages 4193–4202, 2020

    work page 2020

  7. [8]

    Multiscale IoU: A Metric for Evaluation of Salient Object Detection with Fine Structures

    T. Cheng et al. Multiscale iou: A metric for evaluation of salient object detection with fine structures. arXiv preprint arXiv:2105.14572, 2021

    work page Pith review arXiv 2021

  8. [9]

    P. S. Chib and P. Singh. Recent advancements in end-to-end au- tonomous driving using deep learning: A survey. IEEE Transactions on Intelligent Vehicles, 2023

    work page 2023

Show all 55 references
  1. [10]

    Dewangan, T

    V . Dewangan, T. Choudhary, S. Chandhok, S. Priyadarshan, A. Jain, A. K. Singh, S. Srivastava, K. M. Jatavallabhula, and K. M. Krishna. Talk2bev: Language-enhanced bird’s-eye view maps for autonomous driving. arXiv preprint arXiv:2310.02251 , 2023

    arXiv

  2. [11]

    Ding et al

    X. Ding et al. Hilm-d: Towards high-resolution hazard identification and intention prediction for autonomous driving. arXiv preprint arXiv:2310.02933, 2023

    2023 arXiv

  3. [12]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020

    2010 arXiv

  4. [13]

    Dr ´apal, H

    J. Dr ´apal, H. Westermann, and J. Savelka. Using large language models to support thematic analysis in empirical legal studies. arXiv preprint arXiv:2310.18729, 2023

    2023 arXiv

  5. [14]

    Everingham, L

    M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisser- man. The pascal visual object classes (voc) challenge. International Journal of Computer Vision , 88(2):303–338, 2010

    2010

  6. [15]

    Fu et al

    K. Fu et al. Driving with vision-language models: Enhanc- ing autonomous vehicles with multimodal ai. arXiv preprint arXiv:2310.03021, 2023

    2023 arXiv

  7. [16]

    A. Hu, L. Russell, H. Yeo, Z. Murez, G. Fedoseev, A. Kendall, J. Shotton, and G. Corrado. Gaia-1: A generative world model for autonomous driving. arXiv preprint arXiv:2309.17080 , 2023

    2023 arXiv

  8. [17]

    J. Hu, L. Shen, and G. Sun. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132–7141, 2018

    2018

  9. [18]

    P. Hu, A. Huang, J. Dolan, D. Held, and D. Ramanan. Safe local motion planning with self-supervised freespace forecasting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12732–12741, 2021

    2021

  10. [19]

    Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang, L. Lu, X. Jia, Q. Liu, J. Dai, Y . Qiao, and H. Li. Planning-oriented autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023

    2023

  11. [20]

    Huang, W

    Y . Huang, W. Zheng, Y . Zhang, J. Zhou, and J. Lu. Tri-perspective view for vision-based 3d semantic occupancy prediction. In CVPR, pages 9223–9232, 2023

    2023

  12. [21]

    Irons, C

    J. Irons, C. Mason, P. Cooper, S. Sidra, A. Reeson, and C. Paris. Exploring the impacts of chatgpt on future scientific work. 2023

    2023

  13. [22]

    Jiang, S

    B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q. Zhang, W. Liu, C. Huang, and X. Wang. Vad: Vectorized scene representation for efficient autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 8340–8350, 2023

    2023

  14. [23]

    J. Li, K. Han, P. Wang, Y . Liu, and X. Yuan. Anisotropic convolutional networks for 3d semantic scene completion. In CVPR, pages 3351– 3359, 2020

    2020

  15. [24]

    J. Li, D. Li, S. Savarese, and S. Hoi. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597 , 2023

    2023 arXiv

  16. [25]

    Z. Li, Z. Yu, D. Austin, M. Fang, S. Lan, J. Kautz, and J. M. Alvarez. Fb-occ: 3d occupancy prediction based on forward-backward view transformation. arXiv preprint arXiv:2307.01492 , 2023

    2023 arXiv

  17. [26]

    C.-Y . Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pages 74–81. Association for Computational Linguistics, 2004

    2004

  18. [27]

    H. Liu, C. Li, Q. Wu, and Y . J. Lee. Visual instruction tuning. arXiv preprint arXiv:2304.08485, 2023

    2023 arXiv

  19. [28]

    H. Lu, X. Niu, J. Wang, Y . Wang, Q. Hu, J. Tang, Y . Zhang, K. Yuan, B. Huang, Z. Yu, et al. Gpt as psychologist? preliminary evaluations for gpt-4v on visual affective computing. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) workshop, 2024

    2024

  20. [29]

    Mescheder, M

    L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4460–4470, 2019

    2019

  21. [30]

    GPT-4 technical report

    OpenAI. GPT-4 technical report. arXiv preprint arXiv:2303.08774 , 2023

    2023 arXiv

  22. [31]

    Papineni, S

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages 311–318, 2002

    2002

  23. [32]

    C. Qin, A. Zhang, Z. Zhang, J. Chen, M. Yasunaga, and D. Yang. Is chatgpt a general-purpose natural language processing task solver? arXiv preprint arXiv:2302.06476 , 2023

    2023 arXiv

  24. [33]

    A. Rao, J. Kim, M. Kamineni, M. Pang, W. Lie, and M. D. Succi. Evaluating chatgpt as an adjunct for radiologic decision-making. MedRxiv, pages 2023–02, 2023

    2023

  25. [34]

    N. D. Ratliff, J. A. Bagnell, and M. A. Zinkevich. Maximum margin planning. In Proceedings of the 23rd international conference on Machine learning, pages 729–736, 2006

    2006

  26. [35]

    Roldao, R

    L. Roldao, R. de Charette, and A. Verroust-Blondet. Lmscnet: Lightweight multiscale 3d semantic completion. In 3DV, 2020

    2020

  27. [36]

    Ronneberger, P

    O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, pages 234–

  28. [37]

    Y . Shi, B. Paige, P. Torr, et al. Variational mixture-of-experts autoencoders for multi-modal deep generative models. Advances in neural information processing systems , 32, 2019

    2019

  29. [38]

    Shubodh, M

    S. Shubodh, M. Omama, H. Zaidi, U. S. Parihar, and M. Krishna. Lip-loc: Lidar image pretraining for cross-modal localization. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 948–957, 2024

    2024

  30. [39]

    C. Sima, K. Renz, K. Chitta, L. Chen, H. Zhang, C. Xie, P. Luo, A. Geiger, and H. Li. Drivelm: Driving with graph visual question answering. arXiv preprint arXiv:2312.14150 , 2023

    2023 arXiv

  31. [40]

    S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser. Semantic scene completion from a single depth image. In CVPR, pages 1746–1754, 2017

    2017

  32. [41]

    K. Tang, X. Cao, Z. Cao, T. Zhou, E. Li, A. Liu, S. Zou, C. Liu, S. Mei, E. Sizikova, et al. Thma: Tencent hd map ai system for creating hd map annotations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 15585–15593, 2023

    2023

  33. [42]

    G. Team, R. Anil, S. Borgeaud, Y . Wu, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 , 2023

    2023 arXiv

  34. [43]

    The future of driving: Autopilot and full self-driving capabili- ties

    Tesla. The future of driving: Autopilot and full self-driving capabili- ties. https://www.youtube.com/watch?v=ODSJsviD_SU& ab_channel=Tesla, 2022

    2022

  35. [44]

    X. Tian, T. Jiang, L. Yun, Y . Wang, Y . Wang, and H. Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving. arXiv preprint arXiv:2304.14365 , 2023

    2023 arXiv

  36. [45]

    W. Tong, C. Sima, T. Wang, L. Chen, S. Wu, H. Deng, Y . Gu, L. Lu, P. Luo, D. Lin, et al. Scene as occupancy. In ICCV, pages 8406–8415, 2023

    2023

  37. [46]

    W. Tong, C. Sima, T. Wang, L. Chen, S. Wu, H. Deng, Y . Gu, L. Lu, P. Luo, D. Lin, et al. Scene as occupancy. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8406– 8415, 2023

    2023

  38. [47]

    Touvron, L

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  39. [48]

    Vedantam, C

    R. Vedantam, C. L. Zitnick, and D. Parikh. CIDEr: Consensus-based image description evaluation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4566–

  40. [49]

    X. Wang, Z. Zhu, W. Xu, Y . Zhang, Y . Wei, X. Chi, Y . Ye, D. Du, J. Lu, and X. Wang. Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception. In ICCV, pages 17850– 17859, 2023

    2023

  41. [50]

    Y . Wei, L. Zhao, W. Zheng, Z. Zhu, J. Zhou, and J. Lu. Surroundocc: Multi-camera 3d occupancy prediction for autonomous driving. In ICCV, pages 21729–21740, 2023

    2023

  42. [51]

    X. Yan, J. Gao, J. Li, R. Zhang, Z. Li, R. Huang, and S. Cui. Sparse single sweep lidar point cloud segmentation via learning contextual shape priors from scene completion. In AAAI, volume 35, pages 3101– 3109, 2021

    2021

  43. [52]

    W. Zeng, W. Luo, S. Suo, A. Sadat, B. Yang, S. Casas, and R. Urtasun. End-to-end interpretable neural motion planner. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8660–8669, 2019

    2019

  44. [53]

    W. Zeng, X. Ren, T. Su, H. Wang, Y . Liao, Z. Wang, X. Jiang, Z. Yang, K. Wang, X. Zhang, et al. Pangu- α: Large-scale autoregressive pretrained chinese language models with auto-parallel computation. arXiv preprint arXiv:2104.12369 , 2021

    2021 arXiv

  45. [54]

    Zhang, Z

    Y . Zhang, Z. Zhu, and D. Du. Occformer: Dual-path transformer for vision-based 3d semantic occupancy prediction. In ICCV, pages 9433–9443, 2023

    2023

  46. [55]

    Zheng, W

    W. Zheng, W. Chen, Y . Huang, B. Zhang, Y . Duan, and J. Lu. Occworld: Learning a 3d occupancy world model for autonomous driving. arXiv preprint arXiv:2311.16038 , 2023

    2023 arXiv

  47. [56]

    D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny. Minigpt-4: En- hancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592 , 2023

    2023 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.