Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

GaussRender: Learning 3D Occupancy with Gaussian Rendering

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adding a 2D rendering loss to 3D occupancy models improves geometry across three benchmarks.

desk verdict GaussRender is a clean, well-evaluated plug-and-play rendering loss for 3D occupancy that consistently improves geometry-sensitive metrics, though its depth proxy is loosely motivated and the numbers lack error bars. read the letter →

arxiv 2502.05040 v3 pith:ZF4GDT4I submitted 2025-02-07 cs.CV

Loïck Chambon , Eloi Zablocki , Alexandre Boulch , Mickaël Chen , Matthieu Cord This is my paper · ORCID
classification cs.CV
keywords 3DoccupancypredictionGaussiansplattingprojectiveconsistencyrenderinglossautonomousdrivingsemanticscenecompletiondifferentiable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the floating voxels, disjoint surfaces, and misaligned boundaries typical of 3D occupancy predictors come from training losses that treat every voxel independently. It proposes GaussRender, a training-time module that turns both the predicted and ground-truth voxel grids into 3D Gaussians, renders them into camera views with Gaussian splatting, and adds L1 losses on the rendered depth and semantic images. Across three driving benchmarks and three architecturally different occupancy models, the module raises IoU, mIoU, and especially RayIoU, a surface-sensitive metric that rewards correct first-hit depth. The practical point is that a small projective-consistency loss can make existing models produce visually and geometrically coherent scenes without any change at inference time.

What carries the argument

The load-bearing mechanism is differentiable Gaussian splatting applied to voxel grids: each predicted and ground-truth voxel becomes a spherical Gaussian with center at the voxel center, scale tied to voxel size, semantic logits as color, and opacity learned for predictions, then projected with the splatting covariance formula $\Sigma_{2D} = J W \Sigma_{3D} W^T J^T$. Per pixel, the $\alpha$-composited semantic image and depth image are computed as $C_p = \sum_i T_i \alpha_i c_i$ and $D_p = \sum_i T_i \alpha_i d_i$, where $T_i$ is accumulated transmittance. The $L_{2D}$ loss is the L1 distance between predicted and ground-truth renderings, summed over an orthographic bird's-eye view and a dynamically placed virtual camera, and added to the standard 3D loss as $L = L_{3D} + \lambda L_{2D}$.

What would settle it

Train GaussRender on Occ3D-nuScenes with the depth term computed from first-hit depth (the depth at which accumulated transmittance first drops below a threshold) instead of the alpha-composited depth of Eq. (3); if RayIoU gains persist or grow, the alpha-composited proxy is not the load-bearing element, and if the gains disappear, the current depth loss was optimizing a proxy rather than the surface location the benchmark rewards.

Watch

Extended reading notes

Core claim

The central claim is that projective consistency is a missing supervision signal in 3D occupancy learning, and that it can be supplied efficiently by rendering. GaussRender 'gaussianizes' each voxel into a spherical Gaussian with its center and scale fixed from the voxel grid, semantic logits as color, and only opacity learned; renders depth and semantic images from a sensor-like camera and a fixed orthographic bird's-eye camera; and computes L1 losses between the predicted and ground-truth renderings. The authors report consistent gains on SurroundOcc-nuScenes, Occ3D-nuScenes, and SSCBench-KITTI360 for TPVFormer, SurroundOcc, and Symphonies, with the largest relative gains on RayIoU, and state-of-the-art results without lidar supervision and without inference-time modifications.

Load-bearing premise

The depth rendering loss compares alpha-composited, transmittance-weighted depths rather than the depth of the first surface a ray hits, so for thick or multi-layer predictions the loss can be small even when the visible surface is misplaced.

Editorial extensions

If this is right

  • Any existing occupancy model that outputs a voxel grid can be upgraded with GaussRender without architectural changes, as demonstrated on tri-plane, multi-scale voxel, and query-voxel models.
  • Because the largest gains appear on RayIoU and on visible-voxel metrics, the loss specifically reduces surface mislocalization and floating artifacts rather than merely inflating occupancy overlap.
  • Rendering-based supervision works without lidar: GaussRender trained with only the 2D rendering losses outperforms earlier pseudo-label and lidar-projection rendering methods on Occ3D-nuScenes in the 2D-only comparison.
  • The fixed bird's-eye camera and the elevated virtual camera contribute complementary gradients, and the ablations show that each added loss component improves IoU and mIoU.
  • With the projective loss in place, older architectures such as TPVFormer can surpass newer Gaussian-based occupancy models on the tested benchmarks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The alpha-composited depth of Eq. (3) is not identical to the first-occupied-voxel depth that RayIoU measures; for thick or multi-layer predictions, transmittance-weighted depth could be small even when the visible surface is misplaced, so some of the reported RayIoU gain may come from indirect pressure rather than direct optimization of the measured quantity.
  • Inference: Because predicted Gaussians use learned opacity while ground-truth renderings fix opacity at one, part of the semantic rendering loss may be absorbed by adjusting opacity rather than by correcting voxel labels, a degree of freedom absent from the final evaluation.
  • Inference: The camera placement ablations suggest the method's success depends on a modest number of well-chosen viewpoints; random or dynamic camera placement degrades 2D-only training, so the approach is likely to remain robust only when viewpoint selection is constrained.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes GaussRender, a training-only module that converts predicted and ground-truth 3D semantic occupancy grids into spherical Gaussian primitives and renders semantic and depth images from virtual cameras via differentiable Gaussian splatting. The rendered views are compared with L1 losses, which are added to the standard 3D voxel losses. The method is evaluated by integrating it into TPVFormer, SurroundOcc, and Symphonies on SurroundOcc-nuScenes, Occ3D-nuScenes, and SSCBench-KITTI360, reporting consistent improvements in IoU/mIoU and larger gains on the ray-based RayIoU metric, with modest training overhead and no inference-time changes.

Significance. If the reported gains are robust, GaussRender offers a practical, architecture-agnostic loss that improves surface localization in camera-based 3D occupancy prediction and achieves state-of-the-art results on multiple benchmarks with roughly 10% training overhead. The paper ships open-source code, reports detailed per-class results, and includes ablations of loss components, camera placement, Gaussian scale, and loss weight. The main limitations are that the mechanism connecting the alpha-composited depth loss to surface localization is not verified, and the experimental protocol (single runs, possibly uncontrolled baselines) does not yet establish the claimed improvements with sufficient statistical confidence.

major comments (3)
  1. [Sec. 3.3, Eq. (3), Sec. 3.4, Sec. 4.3] The depth rendering loss in Eq. (4) compares the ground-truth first-hit depth (Sec. 3.3 assigns opacity 1 to occupied ground-truth voxels, so the rendered depth is the distance of the first occupied voxel along each ray) with the predicted alpha-composited expected depth Dp = sum_i T_i alpha_i d_i, where predicted opacities are generally below 1. These two quantities are not the same: a prediction with two same-class voxels at d-epsilon and d+epsilon and suitable opacities (e.g., alpha_1=0.5, alpha_2=1) yields approximately zero depth and semantic loss while placing the first surface epsilon meters from the true depth, which is exactly the duplicated-surface artifact that RayIoU is designed to penalize. The paper motivates the depth loss as penalizing floating artifacts and poor surface localization (Sec. 1, Sec. 3.4) and attributes RayIoU gains to it (Sec. 4.3, Table 4), but no analysis is provided showing that the expected-depth proxy tracks first-hit depth. Additionally, Eq. (3) omits the normalization factor sum_i T_i alpha_i, so as written Dp is not a proper expected depth unless the total accumulated opacity is 1. The authors should either justify the proxy formally, provide empirical evidence (e.g., comparing expected-depth maps with first-hit depth maps on validation predictions), or temper the mechanistic claims.
  2. [Sec. 4.1, Tables 1 and 4] The experimental protocol does not ensure a controlled comparison. Section 4.1 states: "By default, we evaluate the original model checkpoints when available; otherwise, we report scores from previous papers if provided or re-train the models for the target dataset. Each model uses the same training settings, following the optimization parameters of [54]." This implies that the baseline numbers in Table 1 and Table 4 may come from official checkpoints or published papers, while the GaussRender-augmented models are retrained with a different (SurroundOcc) optimization recipe. Under this protocol, the reported gains (e.g., TPVFormer +3.75 mIoU on SurroundOcc-nuScenes, +2.65 on Occ3D) could be inflated by the training settings rather than by the proposed loss. The paper should re-train all baselines under identical settings as the GaussRender runs (or explicitly state which entries are re-trained and provide those baselines), and the RayIoU comparison in Table 4 should be computed with the same evaluation code for all entries.
  3. [App. F, Tables 1, 9-11] All main results are single runs without error bars. The paper itself acknowledges in App. F that "The variations by class appear to be due to learning variance, which is why it makes more sense to look at the overall IoU and RayIoU metrics." If class-level scores vary due to learning variance, then the aggregate gains (especially the small ones: +0.11 IoU and +0.26 mIoU for SurroundOcc on KITTI-360, and the negative class-level changes in Table 11) cannot be interpreted as significant without a measure of run-to-run variability. Please report at least 3 seeds with mean and standard deviation for the main tables, or provide a statistical test for the headline claims.
minor comments (6)
  1. [Sec. 3.3] The semantic 'color' is described as 'logits' in the bullet list but as 'probability' in Eq. (3) with c_i in [0,1]^C; please clarify whether a softmax is applied before rendering.
  2. [Eq. (3)] The sums are typeset as "NX i=1" rather than a standard sum; this is a typographical issue that should be corrected.
  3. [Table 2] The caption says "2D-only supervision," but the GaussRender row uses dense voxel ground truth rendered to 2D; the term "2D-only" is potentially misleading and should be redefined (e.g., "rendering-based supervision") to distinguish from self-supervised methods.
  4. [Sec. 4.4.1] The claim that "what matters is to have complementary viewpoints" is based on a 0.4 mIoU difference (26.3 vs 25.9) on a 20% subset; the conclusion is stronger than the evidence supports.
  5. [Table 4] The RayIoU table reports "Previous results are reported from [36]" without stating how the TPVFormer and SurroundOcc baselines were computed; please specify the evaluation protocol for all entries.
  6. [Throughout] Minor typographical inconsistencies: "Kitti360" vs "KITTI360" in Table 1, and "BeV" vs "BEV" variations; please standardize.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GaussRender's rendering loss is directly supervised by ground-truth voxels and its gains are measured against external benchmarks, with no parameter fitted to the reported metrics.

full rationale

The derivation chain is self-contained. The 2D rendering loss (Eq. 4) is an L1 comparison between images rendered from the predicted occupancy and images rendered from the ground-truth voxel grid using the same Gaussian-splatting projection (Eq. 3), so the supervision signal originates from the ground-truth labels rather than from any quantity fitted to the reported IoU/mIoU/RayIoU scores. The paper's central claim—that adding this projective loss improves occupancy models—is tested against external baselines (TPVFormer, SurroundOcc, Symphonies on SurroundOcc-nuScenes, Occ3D-nuScenes, SSCBench-KITTI360), and the RayIoU metric is imported from an external source. Self-citations appear only in the introduction and related work (e.g., PointBEV and motion forecasting references) and are not load-bearing for the method. The Gaussian scale, loss weight, and camera strategy are ablated on validation subsets; tuning hyperparameters on validation is standard practice and does not make a result circular. The depth-loss concern that Eq. (3) alpha-composites expected termination depths for predictions while ground-truth renderings use first-hit depths because GT opacity is 1 is a real proxy-mismatch or correctness risk, not circularity: the loss is exactly the stated L1 between the two renderings, and it does not reduce to its own input by construction. No fitted parameter is renamed as a prediction, no self-citation is used to forbid alternatives, and no known result is merely renamed.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on three engineered choices tuned on validation subsets (loss weight, per-dataset Gaussian scale, and camera placement strategy), plus the assumption that alpha-composited Gaussian rendering of voxel grids is a faithful differentiable proxy for surface depth and semantics. No new physical entities are introduced. The rendering equations come from the Gaussian splatting literature.

free parameters (3)
  • loss weight lambda = 15
    Set from ablation on a 20% Occ3D-nuScenes subset (App. E.2); balances L2D and L3D in the total loss.
  • Gaussian scale s = 0.25 (Occ3D), 0.20 (SurroundOcc), 0.10 (KITTI360)
    Chosen per dataset using the rule 2s = voxel side and confirmed by ablation (App. E.1). Affects rendering density and gradient flow.
  • virtual camera placement hyperparameters = Elevated + Around: elevation with downward tilt and random xy translation up to half scene range
    Camera strategy selected by ablation (Sec. 4.4.1); significant impact under 2D-only supervision and moderate under 2D+3D supervision.
assumptions (4)
  • standard math The Gaussian splatting projection (Eq. 2) and alpha-compositing (Eq. 3) are valid differentiable approximations of image formation.
    Taken from Kerbl et al. [23]; the paper relies on the affine approximation of the projective transformation and the transmittance-weighted sum for color and depth.
  • domain assumption Ground-truth voxel grids (including temporally aggregated occluded voxels) are accurate enough to serve as rendering targets from arbitrary virtual viewpoints.
    The paper itself notes SurroundOcc-nuScenes contains 'temporal artifacts over dynamic objects' (Sec. 4.1), yet these labels are rendered with opacity 1 to supervise virtual cameras.
  • domain assumption Alpha-composited expected depth is an adequate proxy for first-surface depth for the purpose of penalizing floating artifacts.
    Eq. 3 defines depth as a transmittance-weighted average along the ray; the paper motivates the loss with surface localization but never quantifies the gap to first-hit depth.
  • ad hoc to paper The specific 'gaussianization' of voxels (spherical Gaussians, fixed center and scale, learned opacity) preserves enough occupancy information for 2D supervision to improve 3D geometry.
    This is an engineering choice validated only indirectly through ablations (App. E.1); no proof or analysis that it represents the voxel grid faithfully.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GaussRender: Learning 3D Occupancy with Gaussian Rendering." pith.science (2026). https://pith.science/paper/ZF4GDT4I

@misc{pith2026250205040,
  author       = {Pith},
  title        = {Pith review of: GaussRender: Learning 3D Occupancy with Gaussian Rendering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZF4GDT4I}},
  note         = {Machine review of arXiv:2502.05040}
}
read the original abstract

Understanding the 3D geometry and semantics of driving scenes is critical for safe autonomous driving. Recent advances in 3D occupancy prediction have improved scene representation but often suffer from visual inconsistencies, leading to floating artifacts and poor surface localization. Existing voxel-wise losses (e.g., cross-entropy) fail to enforce visible geometric coherence. In this paper, we propose GaussRender, a module that improves 3D occupancy learning by enforcing projective consistency. Our key idea is to project both predicted and ground-truth 3D occupancy into 2D camera views, where we apply supervision. Our method penalizes 3D configurations that produce inconsistent 2D projections, thereby enforcing a more coherent 3D structure. To achieve this efficiently, we leverage differentiable rendering with Gaussian splatting. GaussRender seamlessly integrates with existing architectures while maintaining efficiency and requiring no inference-time modifications. Extensive evaluations on multiple benchmarks (SurroundOcc-nuScenes, Occ3D-nuScenes, SSCBench-KITTI360) demonstrate that GaussRender significantly improves geometric fidelity across various 3D occupancy models (TPVFormer, SurroundOcc, Symphonies), achieving state-of-the-art results, particularly on surface-sensitive metrics such as RayIoU. The code is open-sourced at https://github.com/valeoai/GaussRender.

Figures

Figures reproduced from arXiv: 2502.05040 by the authors.

Figure 1
Figure 1. Comparison of rendered 3D predictions. Standard 3D Occupancy models trained on per-voxel losses result in physically implausible predictions (e.g., floating voxels, poorly localized sur￾faces, highlighted with orange ellipses) that maintain high 3D IoU but fail to produce visually consistent predictions. GaussRen￾der enforces multi-view consistency, eliminating artifacts through learning projective constraints. ages… view at source ↗
Figure 2
Figure 2. Overview of GaussRender. Our module enforces 3D-2D consistency via differentiable Gaussian rendering. First, both predicted and ground-truth voxel grids are ‘gaussianized’ by converting each voxel into a simple spherical Gaussian: the center µ is fixed at the voxel center, the scale s is a simple fixed scaling of the original voxel dimensions, and features are directly transferred as the semantic ‘classes’ c — with … view at source ↗
Figure 3
Figure 3. Arbitrary camera positioning. Virtual cameras can be freely placed in the scene, enabling view constraints in occluded regions. The sensors’ cameras do not provide access to the rear of objects, leading to occlusion issues. GaussRender can place the cameras arbitrarily within the scene, allowing them to be elevated to reach previously hidden areas. space lets GaussRender enhance any 3D occupancy model with a simple … view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Impact of fixed Gaussian scales on 3D mIoU and IoU using TPVFormer [15] trained using only L2D without L3D on 20% of Occ3d-nuScenes [50] validation dataset. (a) σ = 0.06 (b) σ = 0.2 (c) σ = 0.4 (d) σ = 0.06 (e) σ = 0.2 [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Visualization of different Gaussianized voxels for dif￾ferent datasets and scales. The first row represents data from Occ3d-nuScenes [50] and the second and third rows are from SSCBench-Kitti360 [31]. metrics. Theoretically, the optimal size should correlate with the v…
Figure 6
Figure 6. Figure 6: Impact of different the contribution λ of L2D on 3D semantic occupancy performance. The architecture used is TPV￾Former [15]. Models are trained using a combination of L2D and L3D with varying λ values and evaluated on 3D IoU and mIoU. We train and evaluate the models …
Figure 8
Figure 8. Figure 8: Qualitative predictions of a SurroundOcc [54] model trained with GaussRender on the SurroundOcc-nuScenes [54] dataset. We display the six input camera images (top left), the rendered predictions (bottom left), the BeV ground-truth (top right) and BeV prediction (bottom…
Figure 9
Figure 9. Figure 9: Qualitative predictions of a TPVFormer [15] model trained with GaussRender on the Occ3d-nuScenes [50] dataset. We display the six input camera images (top left), the rendered predictions (bottom left), the BeV ground-truth (top right) and BeV prediction (bottom left). …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ODG: Occupancy Prediction Using Dual Gaussians

    cs.CV 2025-06 conditional novelty 6.0 of 10

    ODG uses separate static and dynamic Gaussian query sets, refined coarse-to-fine, plus rendering supervision, and reports state-of-the-art occupancy prediction on Occ3D-nuScenes and Occ3D-Waymo.

  2. VoxDet: Rethinking 3D Semantic Occupancy Prediction as Dense Object Detection

    cs.GR 2025-06 conditional novelty 6.0 of 10

    VoxDet reformulates 3D semantic occupancy prediction as dense object detection by deriving instance-boundary offsets from voxel class labels, and reports new state-of-the-art results on camera and LiDAR benchmarks.

Reference graph

Works this paper leans on

63 extracted references · 60 canonical work pages · cited by 2 Pith papers

  1. [54]

    Surroundocc: Multi-camera 3d occu- pancy prediction for autonomous driving

    Yi Wei, Linqing Zhao, Wenzhao Zheng, Zheng Zhu, Jie Zhou, and Jiwen Lu. Surroundocc: Multi-camera 3d occu- pancy prediction for autonomous driving. ICCV, 2023. 1, 2, 4, 6, 7, 3, 5

    work page 2023

  2. [1]

    Lara: Latents and rays for multi-camera bird’s-eye-view semantic segmen- tation

    Florent Bartoccioni, Eloi Zablocki, Andrei Bursuc, Patrick P´erez, Matthieu Cord, and Karteek Alahari. Lara: Latents and rays for multi-camera bird’s-eye-view semantic segmen- tation. In CoRL, 2022. 1

    work page 2022

  3. [2]

    The lov ´asz-softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks

    Maxim Berman, Amal Rannen Triki, and Matthew B Blaschko. The lov ´asz-softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks. In CVPR, 2018. 1

    work page 2018

  4. [3]

    Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom

    Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. In CVPR, 2020. 6, 1

    work page 2020

  5. [4]

    Monoscene: Monoc- ular 3d semantic scene completion

    Anh-Quan Cao and Raoul de Charette. Monoscene: Monoc- ular 3d semantic scene completion. In CVPR, 2022. 6, 4

    work page 2022

  6. [5]

    Pointbev: A sparse approach to bev predictions

    Lo ¨ıck Chambon, ´Eloi Zablocki, Micka¨el Chen, Florent Bar- toccioni, Patrick P ´erez, and Matthieu Cord. Pointbev: A sparse approach to bev predictions. In CVPR, 2024. 1

    work page 2024

  7. [6]

    Unitraj: A unified framework for scalable vehicle trajectory prediction

    Lan Feng, Mohammadhossein Bahari, Kaouther Mes- saoud Ben Amor, ´Eloi Zablocki, Matthieu Cord, and Alexan- dre Alahi. Unitraj: A unified framework for scalable vehicle trajectory prediction. In ECCV, 2024. 1

    work page 2024

  8. [7]

    Gaussianocc: Fully self-supervised and effi- cient 3d occupancy estimation with gaussian splatting.arXiv preprint arXiv:2408.11447, 2024

    Wanshui Gan, Fang Liu, Hongbin Xu, Ningkai Mo, and Naoto Yokoya. Gaussianocc: Fully self-supervised and effi- cient 3d occupancy estimation with gaussian splatting.arXiv preprint arXiv:2408.11447, 2024. 2, 4

    arXiv 2024

Show all 63 references
  1. [8]

    ViP3D: End-to-end visual trajectory prediction via 3d agent queries

    Junru Gu, Chenxu Hu, Tianyuan Zhang, Xuanyao Chen, Yilun Wang, Yue Wang, and Hang Zhao. ViP3D: End-to-end visual trajectory prediction via 3d agent queries. In CVPR,

  2. [9]

    Fiptr: A simple yet effective transformer framework for future instance prediction in autonomous driving

    Xingtai Gui, Tengteng Huang, Haonan Shao, Haotian Yao, and Chi Zhang. Fiptr: A simple yet effective transformer framework for future instance prediction in autonomous driving. In ECCV, 2024. 1

    2024

  3. [10]

    Harley, Zhaoyuan Fang, Jie Li, Rares Ambrus, and Katerina Fragkiadaki

    Adam W. Harley, Zhaoyuan Fang, Jie Li, Rares Ambrus, and Katerina Fragkiadaki. Simple-BEV: What really matters for multi-sensor bev perception? In ICRA, 2023

    2023

  4. [11]

    FIERY: Future instance segmentation in bird’s-eye view from surround monocular cameras

    Anthony Hu, Zak Murez, Nikhil Mohan, Sof ´ıa Dudas, Jef- frey Hawke, Vijay Badrinarayanan, Roberto Cipolla, and Alex Kendall. FIERY: Future instance segmentation in bird’s-eye view from surround monocular cameras. InICCV, 2021

    2021

  5. [12]

    ST-P3: end-to-end vision-based au- tonomous driving via spatial-temporal feature learning

    Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, and Dacheng Tao. ST-P3: end-to-end vision-based au- tonomous driving via spatial-temporal feature learning. In ECCV, 2022. 1

    2022

  6. [13]

    BEVDet4D: Exploit tempo- ral cues in multi-camera 3d object detection

    Junjie Huang and Guan Huang. BEVDet4D: Exploit tempo- ral cues in multi-camera 3d object detection. arXiv preprint arXiv:2203.17054, 2022. 7

    2022 arXiv

  7. [14]

    Bevdet: High-performance multi-camera 3d object detection in bird-eye-view

    Junjie Huang, Guan Huang, Zheng Zhu, and Dalong Du. Bevdet: High-performance multi-camera 3d object detection in bird-eye-view. CoRR, 2021. 6

    2021

  8. [15]

    Tri-perspective view for vision-based 3d se- mantic occupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Tri-perspective view for vision-based 3d se- mantic occupancy prediction. In CVPR, 2023. 1, 2, 4, 6, 7, 8, 3

    2023

  9. [16]

    Gaussianformer-2: Probabilistic gaussian superposition for efficient 3d occupancy prediction

    Yuanhui Huang, Amonnut Thammatadatrakoon, Wenzhao Zheng, Yunpeng Zhang, Dalong Du, and Jiwen Lu. Gaussianformer-2: Probabilistic gaussian superposition for efficient 3d occupancy prediction. CoRR, 2024. 1, 2, 6, 7, 4

    2024

  10. [17]

    Selfocc: Self-supervised vision-based 3d oc- cupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Borui Zhang, Jie Zhou, and Jiwen Lu. Selfocc: Self-supervised vision-based 3d oc- cupancy prediction. In CVPR, 2024. 2, 3, 4

    2024

  11. [18]

    Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction. In ECCV,

  12. [19]

    Sym- phonize 3d semantic scene completion with contextual in- stance queries

    Haoyi Jiang, Tianheng Cheng, Naiyu Gao, Haoyang Zhang, Tianwei Lin, Wenyu Liu, and Xinggang Wang. Sym- phonize 3d semantic scene completion with contextual in- stance queries. In CVPR, 2024. 1, 2, 4, 6, 7

    2024

  13. [20]

    Gausstr: Foundation model-aligned gaussian transformer for self-supervised 3d spatial understanding

    Haoyi Jiang, Liu Liu, Tianheng Cheng, Xinjie Wang, Tian- wei Lin, Zhizhong Su, Wenyu Liu, and Xinggang Wang. Gausstr: Foundation model-aligned gaussian transformer for self-supervised 3d spatial understanding. CoRR, 2024. 4

    2024

  14. [21]

    Far3d: Expanding the horizon for surround-view 3d object detec- tion

    Xiaohui Jiang, Shuailin Li, Yingfei Liu, Shihao Wang, Fan Jia, Tiancai Wang, Lijin Han, and Xiangyu Zhang. Far3d: Expanding the horizon for surround-view 3d object detec- tion. In EAAI, 2024. 1

    2024

  15. [22]

    Neu- ral 3d mesh renderer

    Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada. Neu- ral 3d mesh renderer. In CVPR, 2018. 3

    2018

  16. [23]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics ,

  17. [24]

    Point cloud forecasting as a proxy for 4d occupancy forecasting

    Tarasha Khurana, Peiyun Hu, David Held, and Deva Ra- manan. Point cloud forecasting as a proxy for 4d occupancy forecasting. In CPVR, 2023. 2

    2023

  18. [25]

    Lapred: Lane-aware prediction of multi- modal future trajectories of dynamic agents

    ByeoungDo Kim, Seong Hyeon Park, Seokhwan Lee, Elbek Khoshimjonov, Dongsuk Kum, Junsoo Kim, Jeong Soo Kim, and Jun Won Choi. Lapred: Lane-aware prediction of multi- modal future trajectories of dynamic agents. In CVPR, 2021. 1

    2021

  19. [26]

    Hierarchical temporal context learning for camera-based semantic scene comple- tion

    Bohan Li, Jiajun Deng, Wenyao Zhang, Zhujin Liang, Da- long Du, Xin Jin, and Wenjun Zeng. Hierarchical temporal context learning for camera-based semantic scene comple- tion. In ECCV, 2024. 2

    2024

  20. [27]

    Viewformer: Exploring spatiotem- poral modeling for multi-view 3d occupancy perception via view-guided transformers

    Jinke Li, Xiao He, Chonghua Zhou, Xiaoqiang Cheng, Yang Wen, and Dan Zhang. Viewformer: Exploring spatiotem- poral modeling for multi-view 3d occupancy perception via view-guided transformers. In ECCV, 2024. 2

    2024

  21. [28]

    Bevstereo: Enhancing depth estimation in multi-view 3d object detection with temporal stereo

    Yinhao Li, Han Bao, Zheng Ge, Jinrong Yang, Jianjian Sun, and Zeming Li. Bevstereo: Enhancing depth estimation in multi-view 3d object detection with temporal stereo. In EAAI, 2023. 6

    2023

  22. [29]

    Bevdepth: Acquisition of reliable depth for multi-view 3d object detec- tion

    Yinhao Li, Zheng Ge, Guanyi Yu, Jinrong Yang, Zengran Wang, Yukang Shi, Jianjian Sun, and Zeming Li. Bevdepth: Acquisition of reliable depth for multi-view 3d object detec- tion. In EAAI, 2023. 1

    2023

  23. [30]

    Choy, Chaowei Xiao, Jos ´e M

    Yiming Li, Zhiding Yu, Christopher B. Choy, Chaowei Xiao, Jos ´e M. ´Alvarez, Sanja Fidler, Chen Feng, and An- ima Anandkumar. V oxformer: Sparse voxel transformer for camera-based 3d semantic scene completion. In CVPR,

  24. [31]

    Ss- cbench: A large-scale 3d semantic scene completion bench- mark for autonomous driving

    Yiming Li, Sihang Li, Xinhao Liu, Moonjun Gong, Kenan Li, Nuo Chen, Zijun Wang, Zhiheng Li, Tao Jiang, Fisher Yu, Yue Wang, Hang Zhao, Zhiding Yu, and Chen Feng. Ss- cbench: A large-scale 3d semantic scene completion bench- mark for autonomous driving. In IROS, 2024. 2, 4, 6, 1, 3

    2024

  25. [32]

    Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers

    Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chong- hao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers. In ECCV, 2022. 1, 6, 7, 4

    2022

  26. [33]

    ´Alvarez

    Zhiqi Li, Zhiding Yu, David Austin, Mingsheng Fang, Shiyi Lan, Jan Kautz, and Jos ´e M. ´Alvarez. FB-OCC: 3d occu- pancy prediction based on forward-backward view transfor- mation. CoRR, 2023. 1, 7

    2023

  27. [34]

    KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d

    Yiyi Liao, Jun Xie, and Andreas Geiger. KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d. PAMI, 2022. 6, 7, 1

    2022

  28. [35]

    Sparse4d: Multi-view 3d object detection with sparse spatial-temporal fusion

    Xuewu Lin, Tianwei Lin, Zixiang Pei, Lichao Huang, and Zhizhong Su. Sparse4d: Multi-view 3d object detection with sparse spatial-temporal fusion. CoRR, 2022. 1

    2022

  29. [36]

    Fully sparse 3d occupancy prediction

    Haisong Liu, Haiguang Wang, Yang Chen, Zetong Yang, Jia Zeng, Li Chen, and Limin Wang. Fully sparse 3d occupancy prediction. In ECCV, 2024. 1, 2, 4, 7

    2024

  30. [37]

    Pa- parazzi: surface editing by way of multi-view image process- ing

    Hsueh-Ti Derek Liu, Michael Tao, and Alec Jacobson. Pa- parazzi: surface editing by way of multi-view image process- ing. ACM, 2018. 3

    2018

  31. [38]

    PETR: position embedding transformation for multi-view 3d object detection

    Yingfei Liu, Tiancai Wang, Xiangyu Zhang, and Jian Sun. PETR: position embedding transformation for multi-view 3d object detection. In ECCV, 2022. 1

    2022

  32. [39]

    Oc- treeocc: Efficient and multi-granularity occupancy predic- tion using octree queries

    Yuhang Lu, Xinge Zhu, Tai Wang, and Yuexin Ma. Oc- treeocc: Efficient and multi-granularity occupancy predic- tion using octree queries. CoRR, 2023. 2

    2023

  33. [40]

    COTR: compact occupancy trans- former for vision-based 3d occupancy prediction

    Qihang Ma, Xin Tan, Yanyun Qu, Lizhuang Ma, Zhizhong Zhang, and Yuan Xie. COTR: compact occupancy trans- former for vision-based 3d occupancy prediction. In CVPR,

  34. [41]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. In ECCV, 2020. 3

    2020

  35. [42]

    Atlas: End- to-end 3d scene reconstruction from posed images

    Zak Murez, Tarrence van As, James Bartolozzi, Ayan Sinha, Vijay Badrinarayanan, and Andrew Rabinovich. Atlas: End- to-end 3d scene reconstruction from posed images. InECCV,

  36. [43]

    Renderocc: Vision-centric 3d occupancy prediction with 2d rendering supervision

    Mingjie Pan, Jiaming Liu, Renrui Zhang, Peixiang Huang, Xiaoqi Li, Hongwei Xie, Bing Wang, Li Liu, and Shanghang Zhang. Renderocc: Vision-centric 3d occupancy prediction with 2d rendering supervision. In ICRA, 2024. 2, 3, 4, 6, 7

    2024

  37. [44]

    Motion transformer with global intention localization and lo- cal movement refinement

    Shaoshuai Shi, Li Jiang, Dengxin Dai, and Bernt Schiele. Motion transformer with global intention localization and lo- cal movement refinement. In NeuRIPS, 2022. 1

    2022

  38. [45]

    MTR++: multi-agent motion prediction with symmetric scene modeling and guided intention querying

    Shaoshuai Shi, Li Jiang, Dengxin Dai, and Bernt Schiele. MTR++: multi-agent motion prediction with symmetric scene modeling and guided intention querying. TPAMI,

  39. [46]

    Unified spatio-temporal tri-perspective view representation for 3d se- mantic occupancy prediction, 2024

    Sathira Silva, Savindu Bhashitha Wannigama, Gihan Jayati- laka, Muhammad Haris Khan, and Roshan Ragel. Unified spatio-temporal tri-perspective view representation for 3d se- mantic occupancy prediction, 2024. 2

    2024

  40. [47]

    Occfeat: Self-supervised occupancy feature prediction for pretraining BEV segmentation networks

    Sophia Sirko-Galouchenko, Alexandre Boulch, Spyros Gi- daris, Andrei Bursuc, Anton ´ın V obeck´y, Patrick P ´erez, and Renaud Marlet. Occfeat: Self-supervised occupancy feature prediction for pretraining BEV segmentation networks. In CVPR Workshop, 2024. 1

    2024

  41. [48]

    Sudre, Wenqi Li, Tom Vercauteren, S ´ebastien Ourselin, and M

    Carole H. Sudre, Wenqi Li, Tom Vercauteren, S ´ebastien Ourselin, and M. Jorge Cardoso. Generalised dice overlap as a deep learning loss function for highly unbalanced seg- mentations. In DLMIA, 2017. 1

    2017

  42. [49]

    Gsrender: Dedupli- cated occupancy prediction via weakly supervised 3d gaus- sian splatting

    Qianpu Sun, Changyong Shu, Sifan Zhou, Zichen Yu, Yan Chen, Dawei Yang, and Yuan Chun. Gsrender: Dedupli- cated occupancy prediction via weakly supervised 3d gaus- sian splatting. CoRR, 2024. 2, 3, 4, 6, 7

    2024

  43. [50]

    Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving

    Xiaoyu Tian, Tao Jiang, Longfei Yun, Yucheng Mao, Huitong Yang, Yue Wang, Yilun Wang, and Hang Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving. In NeuRIPS, 2023. 2, 4, 6, 7, 8, 1, 3

    2023

  44. [51]

    Mambaocc: Visual state space model for bev-based occupancy prediction with local adaptive reordering

    Yonglin Tian, Songlin Bai, Zhiyao Luo, Yutong Wang, Yisheng Lv, and Fei-Yue Wang. Mambaocc: Visual state space model for bev-based occupancy prediction with local adaptive reordering. CoRR, 2024. 2

    2024

  45. [52]

    Pop- 3d: Open-vocabulary 3d occupancy prediction from images

    Antonin V obecky, Oriane Sim´eoni, David Hurych, Spyridon Gidaris, Andrei Bursuc, Patrick P´erez, and Josef Sivic. Pop- 3d: Open-vocabulary 3d occupancy prediction from images. In NeuRIPS, 2023. 1

    2023

  46. [53]

    Occsora: 4d occupancy generation models as world simulators for au- tonomous driving

    Lening Wang, Wenzhao Zheng, Yilong Ren, Han Jiang, Zhiyong Cui, Haiyang Yu, and Jiwen Lu. Occsora: 4d occupancy generation models as world simulators for au- tonomous driving. CoRR, 2024. 2

    2024

  47. [55]

    3d semantic scene completion and occupancy prediction for autonomous driving: A survey

    Guixing Xu, Wei Liu, Zuotao Ning, Qixi Zhao, Shuai Cheng, and Jiwei Nie. 3d semantic scene completion and occupancy prediction for autonomous driving: A survey. InCAIT, 2023. 2

    2023

  48. [56]

    Towards motion forecasting with real-world perception inputs: Are end-to-end approaches competitive? In ICRA, 2024

    Yihong Xu, Lo ¨ıck Chambon, ´Eloi Zablocki, Micka ¨el Chen, Alexandre Alahi, Matthieu Cord, and Patrick P´erez. Towards motion forecasting with real-world perception inputs: Are end-to-end approaches competitive? In ICRA, 2024. 1

    2024

  49. [57]

    PPT: pre-training with pseudo-labeled trajectories for motion forecasting

    Yihong Xu, Yuan Yin, Tuan-Hung Vu, Alexandre Boulch, ´Eloi Zablocki, and Matthieu Cord. PPT: pre-training with pseudo-labeled trajectories for motion forecasting. CoRR,

  50. [58]

    Driving in the occupancy world: Vision-centric 4d occupancy forecast- ing and planning via world models for autonomous driving

    Yu Yang, Jianbiao Mei, Yukai Ma, Siliang Du, Wenqing Chen, Yijie Qian, Yuxiang Feng, and Yong Liu. Driving in the occupancy world: Vision-centric 4d occupancy forecast- ing and planning via world models for autonomous driving. CoRR, 2024. 2

    2024

  51. [59]

    Cvt-occ: Cost volume temporal fusion for 3d occupancy prediction

    Zhangchen Ye, Tao Jiang, Chenfeng Xu, Yiming Li, and Hang Zhao. Cvt-occ: Cost volume temporal fusion for 3d occupancy prediction. In ECCV, 2024. 2 10

    2024

  52. [60]

    Agentformer: Agent-aware transformers for socio-temporal multi-agent forecasting

    Ye Yuan, Xinshuo Weng, Yanglan Ou, and Kris Kitani. Agentformer: Agent-aware transformers for socio-temporal multi-agent forecasting. In ICCV, 2021. 1

    2021

  53. [61]

    Occformer: Dual-path transformer for vision-based 3d semantic occu- pancy prediction

    Yunpeng Zhang, Zheng Zhu, and Dalong Du. Occformer: Dual-path transformer for vision-based 3d semantic occu- pancy prediction. In ICCV, 2023. 6, 4

    2023

  54. [62]

    Lowrankocc: Tensor decomposition and low-rank recovery for vision-based 3d semantic occupancy prediction

    Linqing Zhao, Xiuwei Xu, Ziwei Wang, Yunpeng Zhang, Borui Zhang, Wenzhao Zheng, Dalong Du, Jie Zhou, and Jiwen Lu. Lowrankocc: Tensor decomposition and low-rank recovery for vision-based 3d semantic occupancy prediction. In CVPR, 2024. 2

    2024

  55. [63]

    Unim2ae: Multi- modal masked autoencoders with unified 3d representation for 3d perception in autonomous driving

    Jian Zou, Tianyu Huang, Guanglei Yang, Zhenhua Guo, Tao Luo, Chun-Mei Feng, and Wangmeng Zuo. Unim2ae: Multi- modal masked autoencoders with unified 3d representation for 3d perception in autonomous driving. In ECCV, 2024. 1 11 GaussRender: Learning 3D Occupancy with Gaussian ...

    2024

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.