REVIEW 3 major objections 5 minor 95 references
Monocular Semantic Scene Completion via Masked Recurrent Networks
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Two-stage masked recurrent refinement sets new state-of-the-art in monocular 3D scene completion.
desk verdict A useful two-stage architecture with strong main-table ablations, undercut by a contradictory appendix table and a confounded SOTA comparison. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Masked Sparse Gated Recurrent Unit (MS-GRU), a gated recurrent unit that applies submanifold sparse convolution to masked inputs when computing its update and reset gates, and sparse convolution when computing the candidate hidden state, so every recurrent step touches only voxels the current mask marks as occupied. It is driven by two supporting mechanisms. The Distance Attention Projection (DAP) weights sight-projected 2D features by an attention value along the line of sight that is 1 at the estimated surface, decays beyond it, and is lower in front of it, reducing projection errors in occluded regions; the Mask Updating Module revises the occupancy mask from the current semantic prediction by adding the most-confident occupied voxels and removing the most-confident empty ones. The mask is supervised by a sequential weighted binary cross-entropy loss at every step.
What would settle it
The paper's own appendix Table 6 reports that removing the mask updating module, mask initialization, or mask loss each improves SC-IoU over the full model, contradicting the main-text ablation; re-running the released code under one protocol would show which table a fresh run reproduces, and that single check would settle whether the mask-related components contribute the credited gains.
Extended reading notes
Core claim
On its own terms, the paper's central claim is that the two-stage decomposition MonoMRN effectively solves both indoor and outdoor monocular semantic scene completion and outperforms published single-stage methods. The coarse stage uses a pretrained 2D encoder and a learned depth estimate to project features into 3D, then a 3D network built from anisotropic convolution blocks produces the initial occupancy and semantics together with an initial mask. The Masked Recurrent Network then runs several iterations of the Masked Sparse Gated Recurrent Unit, which computes update and reset gates with submanifold sparse convolutions over masked voxels, so each step rewrites only occupied regions; the mask is revised by a mask updating module, and the distance attention projection feeds in features whose weights fall off with distance from the estimated observed surface. On NYUv2 the method reports 53.16 scene-completion IoU and 30.73 semantic mIoU, and on SemanticKITTI 42.0 SC-IoU and 13.8 SSC-mIoU, with ablations crediting the recurrent components for part of the gain and robustness experiments showing less degradation under four types of disturbance.
Load-bearing premise
The central assumption is that the gains over prior methods are produced by the recurrent refinement and its components, rather than by the stronger coarse stage that already uses learned depth, a ResNet-50 backbone, and a heavier 3D network than the published baselines.
Editorial extensions
If this is right
- Iterative masked refinement lets a monocular model recover occluded regions and fine object details that a single forward pass misses, as shown by the progressive improvement from coarse output through two recurrent iterations.
- The sparse design makes this refinement cheap: MS-GRU uses the same parameter count as a standard GRU but roughly a third of the multiply-accumulate operations (52.44 G versus 171.99 G).
- Replacing the distance attention projection with either surface or sight projection lowers both metrics, so weighting features by distance to the observed surface is the mechanism that lets occluded voxels get useful features without the noise of raw sight projection.
- Because the same components work for both indoor and outdoor scenes, the two-stage recipe transfers across environments without task-specific changes, and the robustness tests indicate the recurrent stage limits degradation under common disturbances.
Reading between the lines
- A decisive way to separate the recurrent refinement from the depth prior is to freeze the coarse stage and swap only the refiner, or to replace the learned depth with oracle depth; the paper does not report this decomposition, and it would clarify how much of the margin over MonoScene comes from refinement rather than from the stronger depth-based coarse stage.
- Because the distance attention weights in equation (6) are fixed by hand, making them a learned function of predicted depth uncertainty would give the model an explicit mechanism to discount unreliable projections, which the fog and darkness robustness results suggest would help.
- The same masked recurrent pattern could be carried to stereo or temporal occupancy prediction, where the coarse stage is stronger and the voxel grids are larger, making the sparsity savings of MS-GRU even more valuable.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MonoMRN, a two-stage monocular semantic scene completion framework that first produces a coarse MSSC estimate using a ResNet-50 encoder, AdaBins depth estimation, and an AIC-based 3D network, and then refines it with a Masked Recurrent Network. The recurrent stage comprises three novel components: the Masked Sparse Gated Recurrent Unit (MS-GRU), which applies an iteratively updated occupancy mask and sparse convolutions; the Distance Attention Projection (DAP), which weights sight-projected 2D features by distance to the estimated surface; and a Mask Updating Module that sequentially refines the occupancy mask. Experiments on NYUv2 and SemanticKITTI report state-of-the-art SC-IoU and SSC-mIoU, component ablations, efficiency comparisons, and a robustness study under dark, motion, brightness, and fog perturbations.
Significance. If the reported results and ablations are correct, the paper makes a credible contribution: a two-stage decomposition of monocular semantic scene completion is a reasonable architectural direction, the sparse masked recurrent design offers a measurable efficiency benefit (Table 5, 52.44 G MACs vs 171.99 G for standard GRU), and the public code release supports reproducibility. The reported absolute gains over MonoScene on NYUv2 are substantial. However, the paper's central attribution of these gains to the Masked Recurrent Network is currently undermined by an internal contradiction between the main-text and appendix ablations, and the SOTA comparison is partially confounded by the stronger coarse stage. These issues must be resolved before the claims can be accepted as stated.
major comments (3)
- [Table 3 vs Appendix Table 6] The two ablation tables directly contradict each other on the effect of the mask-related components. Main-text Table 3 shows that adding the Mask Updating Module to '+Distance Attention Projection' improves SC-IoU from 51.86 to 53.16 and SSC-mIoU from 30.11 to 30.73. Appendix Table 6, however, reports 'With Mask Updating Module' as 52.33/30.11 and 'W/O Mask Updating Module' as 53.16/30.73, which is the exact opposite conclusion. The same inversion appears for 'With/W/O Mask Initialization' and 'With/W/O Mask Loss', where the 'W/O' rows equal the full-model numbers. As printed, the appendix table implies the mask components either hurt or have no effect, directly contradicting Section 4.4 and Section 8. The authors must clarify which table corresponds to the actual experimental configuration, correct the mislabeled/misconfigured entries, and ensure the main-text claims are supported by the ablation data.
- [Section 4.2 and Table 3 (baseline confound)] The SOTA comparisons do not isolate the contribution of the Masked Recurrent Network from the stronger coarse stage. Table 3 shows that the coarse-stage baseline already reaches 48.23 SC-IoU and 27.47 SSC-mIoU on NYUv2, versus MonoScene's 42.51 and 26.94 in Table 1. Because the coarse stage uses AdaBins depth, a ResNet-50 backbone, and an AIC-based 3D network, part of the reported margin over prior monocular methods may come from these components rather than from MS-GRU, DAP, or the mask updating mechanism. To support the claim that the recurrent refinement is the source of the SOTA improvement, the authors should provide a comparison in which the coarse stage is held comparable to prior work, or at least report the performance of the coarse stage alone against MonoScene and NDC-Scene under the same evaluation protocol.
- [Figure 7 and Section 4.3] The robustness analysis does not currently support the strength of the claim that the Masked Recurrent Network 'enhances resilience.' Figure 7 reports single numbers for each perturbation level without error bars or repeated-seed variance, and the text does not define the perturbation protocol (e.g., exact darkness/motion/brightness/fog parameters, whether the same perturbed inputs are used for both models, and whether SemanticKITTI results are on the test or validation split). Since robustness is presented as an explicit contribution, the authors should add a precise protocol, report mean and standard deviation over multiple runs, and state the evaluation split.
minor comments (5)
- [Section 8 / Table 6] The sentence 'Mask initialization could obtain 1.11% performance gain' is ambiguous: the SC-IoU difference between 'With Mask Initialization' (51.26) and 'W/O Mask Initialization' (53.16) is 1.90 points, while the SSC-mIoU difference is 1.11 points. Please state which metric is being quoted and use consistent terminology for percentage-point gains.
- [Equation (6)] The piecewise definition of wd appears to contain an inconsistency: the text says 'd < d′' denotes the region before the observed surface, but the third case is written as '0.5, δ < d < d′'. Please clarify the intended domain and how δ relates to the RMS of the predicted depth.
- [Table 2] In the SemanticKITTI comparison, several rows are listed with 'M' (monocular) but are actually described elsewhere in the table as stereo or temporal methods (e.g., 'HTCL-M [40]' is marked 'S&T' in the input column). Please double-check the input labels and the hyphen placeholders in the class-wise columns so that each method's input modality is unambiguous.
- [Section 4.2] The text reports 'an 8.99% IoU increase in SC and a 1.60% mIoU improvement in SSC' relative to 'state-of-the-art methods' on NYUv2. The 8.99-point SC-IoU improvement appears to be relative to NDC-Scene (44.17), while the 1.60-point SSC-mIoU improvement appears to be relative to NDC-Scene (29.03); please state the reference method explicitly to avoid ambiguity.
- [Related Work] There is a typo in the Related Work section where the reference list appears as '[50, 50, 50, 74, 80]'; this should be corrected to the intended distinct references.
Circularity Check
No significant circularity: MonoMRN's SOTA claims rest on external benchmarks and component ablations; the Appendix Table 6 inconsistency is a correctness issue, not a circular derivation.
full rationale
The paper's central derivation chain is method construction followed by external evaluation: coarse MSSC produces h0 and m0, the Masked Recurrent Network iterates via MS-GRU equations (1)-(4), Distance Attention Projection equation (6), mask updating, and losses (8)-(10). The reported gains are measured against published baselines on NYUv2 and SemanticKITTI, and none of the proposed components is defined in terms of the target metrics. Hyperparameters such as the mask threshold 0.6, top-K 5, loss weights 0.8/0.6, and the depth-RMS-based delta are standard validation choices, not fitted quantities renamed as predictions. The paper does cite the authors' own prior work (e.g., FFNet, NUC-Net, HASSC-adjacent works), but these citations are contextual and not load-bearing for the central claim; the only externally invoked loss, Lscal, is attributed to MonoScene [7], which is not the present authors' work. No uniqueness theorem or ansatz is imported from the authors' own prior publications. The Appendix Table 6 contradiction with main-text Table 3, where removing mask updating, mask initialization, or mask loss yields the full-model numbers, is a serious internal consistency and attribution problem, and the coarse stage's stronger depth/backbone/3D network is a real confound for the SOTA margin; however, these are empirical correctness and confound concerns, not circular reductions of the prediction to its inputs. The derivation is therefore self-contained against external benchmarks, and the circularity score is 0.
Assumptions & free parameters
free parameters (6)
- mask initialization threshold t =
0.6
- mask updating top-K count =
5
- sequential MSSC loss decay gamma =
0.8
- sequential mask loss decay gamma =
0.6
- distance attention weights for pre-surface voxels and delta =
0.5 and RMS-of-depth dependent
- number of recurrent iterations =
2
assumptions (4)
- domain assumption The NYUv2 and SemanticKITTI benchmarks and the SSCNet evaluation protocol are valid proxies for scene completion quality.
- domain assumption The pretrained AdaBins depth estimator provides depth maps accurate enough for the 2D-to-3D projection in the coarse stage.
- domain assumption The occupancy mask ground truth obtained by summing all non-empty classes is a useful supervisory signal.
- domain assumption The recurrent refinement converges stably under the sequential cross-entropy and mask losses.
invented entities (3)
-
Masked Sparse Gated Recurrent Unit (MS-GRU)
independent evidence
-
Distance Attention Projection (DAP)
independent evidence
-
Mask Updating Module
independent evidence
Cite this review
Pith. "Pith review of Monocular Semantic Scene Completion via Masked Recurrent Networks." pith.science (2026). https://pith.science/paper/ZWGNLJ7I
@misc{pith2026250717661,
author = {Pith},
title = {Pith review of: Monocular Semantic Scene Completion via Masked Recurrent Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZWGNLJ7I}},
note = {Machine review of arXiv:2507.17661}
}
read the original abstract
Monocular Semantic Scene Completion (MSSC) aims to predict the voxel-wise occupancy and semantic category from a single-view RGB image. Existing methods adopt a single-stage framework that aims to simultaneously achieve visible region segmentation and occluded region hallucination, while also being affected by inaccurate depth estimation. Such methods often achieve suboptimal performance, especially in complex scenes. We propose a novel two-stage framework that decomposes MSSC into coarse MSSC followed by the Masked Recurrent Network. Specifically, we propose the Masked Sparse Gated Recurrent Unit (MS-GRU) which concentrates on the occupied regions by the proposed mask updating mechanism, and a sparse GRU design is proposed to reduce the computation cost. Additionally, we propose the distance attention projection to reduce projection errors by assigning different attention scores according to the distance to the observed surface. Experimental results demonstrate that our proposed unified framework, MonoMRN, effectively supports both indoor and outdoor scenes and achieves state-of-the-art performance on the NYUv2 and SemanticKITTI datasets. Furthermore, we conduct robustness analysis under various disturbances, highlighting the role of the Masked Recurrent Network in enhancing the model's resilience to such challenges. The source code is publicly available.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
Learning representations and generative models for 3d point clouds
Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. Learning representations and generative models for 3d point clouds. In International Conference on Machine Learning, pages 40–49. PMLR, 2018
2018
-
[2]
Rangevit: Towards vision transformers for 3d semantic segmentation in au- tonomous driving
Angelika Ando, Spyros Gidaris, Andrei Bursuc, Gilles Puy, Alexandre Boulch, and Renaud Marlet. Rangevit: Towards vision transformers for 3d semantic segmentation in au- tonomous driving. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 5240–5250, 2023
2023
-
[3]
Se- mantickitti: A dataset for semantic scene understanding of lidar sequences
Jens Behley, Martin Garbade, Andres Milioto, Jan Quen- zel, Sven Behnke, Cyrill Stachniss, and Jurgen Gall. Se- mantickitti: A dataset for semantic scene understanding of lidar sequences. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision , pages 9297–9307, 2019
2019
-
[4]
Adabins: Depth estimation using adaptive bins
Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4009–4018, 2021
2021
-
[5]
Dynamiccity: Large-scale 4d oc- cupancy generation from dynamic scenes
Hengwei Bian, Lingdong Kong, Haozhe Xie, Liang Pan, Yu Qiao, and Ziwei Liu. Dynamiccity: Large-scale 4d oc- cupancy generation from dynamic scenes. In International Conference on Learning Representations, 2025
2025
-
[6]
Also: Automotive lidar self- supervision by occupancy estimation
Alexandre Boulch, Corentin Sautier, Bj ¨orn Michele, Gilles Puy, and Renaud Marlet. Also: Automotive lidar self- supervision by occupancy estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13455–13465, 2023
2023
-
[7]
Monoscene: Monoc- ular 3d semantic scene completion
Anh-Quan Cao and Raoul De Charette. Monoscene: Monoc- ular 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3991–4001, 2022
2022
-
[8]
Scenerf: Self- supervised monocular 3d scene reconstruction with radiance fields
Anh-Quan Cao and Raoul De Charette. Scenerf: Self- supervised monocular 3d scene reconstruction with radiance fields. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 9387–9398, 2023
2023
Show all 95 references
-
[9]
Pcam: Product of cross-attention matrices for rigid registration of point clouds
Anh-Quan Cao, Gilles Puy, Alexandre Boulch, and Renaud Marlet. Pcam: Product of cross-attention matrices for rigid registration of point clouds. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 13229– 13238, 2021
2021
-
[10]
Pasco: Urban 3d panoptic scene completion with uncertainty aware- ness
Anh-Quan Cao, Angela Dai, and Raoul de Charette. Pasco: Urban 3d panoptic scene completion with uncertainty aware- ness. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 14554–14564, 2024
2024
-
[11]
Building a strong pre- training baseline for universal 3d large-scale perception
Haoming Chen, Zhizhong Zhang, Yanyun Qu, Ruixin Zhang, Xin Tan, and Yuan Xie. Building a strong pre- training baseline for universal 3d large-scale perception. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 19925–19935, 2024
2024
-
[12]
Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning
Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, Wei Liu, and Tat-Seng Chua. Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017
2017
-
[13]
Towards label-free scene understanding by vision foundation models
Runnan Chen, Youquan Liu, Lingdong Kong, Nenglun Chen, Xinge Zhu, Yuexin Ma, Tongliang Liu, and Wenping Wang. Towards label-free scene understanding by vision foundation models. In Advances in Neural Information Pro- cessing Systems, pages 75896–75910, 2023
2023
-
[14]
Clip2scene: Towards label-efficient 3d scene understanding by clip
Runnan Chen, Youquan Liu, Lingdong Kong, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao, and Wen- ping Wang. Clip2scene: Towards label-efficient 3d scene understanding by clip. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 7...
2023
-
[15]
3d sketch-aware semantic scene comple- tion via semi-supervised structure prior
Xiaokang Chen, Kwan-Yee Lin, Chen Qian, Gang Zeng, and Hongsheng Li. 3d sketch-aware semantic scene comple- tion via semi-supervised structure prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4193–4202, 2020
2020
-
[16]
S3cnet: A sparse semantic scene completion net- work for lidar point cloud
Ran Cheng, Christopher Agia, Yuan Ren, Xinhai Li, and Liu Bingbing. S3cnet: A sparse semantic scene completion net- work for lidar point cloud. InConference on Robot Learning, pages 2148–2161, 2021
2021
-
[17]
Af2-s3net: Attentive feature fusion with adap- tive feature selection for sparse semantic segmentation net- work
Ran Cheng, Ryan Razani, Ehsan Taghavi, Enxu Li, and Bingbing Liu. Af2-s3net: Attentive feature fusion with adap- tive feature selection for sparse semantic segmentation net- work. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 12...
2021
-
[18]
4d spatio-temporal convnets: Minkowski convolutional neu- ral networks
Christopher Choy, JunYoung Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neu- ral networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3075– 3084, 2019
2019
-
[19]
3d-r2n2: A unified approach for single and multi-view 3d object reconstruction
Christopher B Choy, Danfei Xu, JunYoung Gwak, Kevin Chen, and Silvio Savarese. 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In European Conference on Computer Vision , pages 628–644. Springer, 2016
2016
-
[20]
Buol: A bottom-up framework with occupancy-aware lifting for panoptic 3d scene reconstruction from a single image
Tao Chu, Pan Zhang, Qiong Liu, and Jiaqi Wang. Buol: A bottom-up framework with occupancy-aware lifting for panoptic 3d scene reconstruction from a single image. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 4937–4946, 2023
2023
-
[21]
Panoptic 3d scene reconstruction from a single rgb image
Manuel Dahnert, Ji Hou, Matthias Nießner, and Angela Dai. Panoptic 3d scene reconstruction from a single rgb image. Advances in Neural Information Processing Systems , 34: 8282–8293, 2021
2021
-
[22]
V ote3deep: Fast ob- ject detection in 3d point clouds using efficient convolu- tional neural networks
Martin Engelcke, Dushyant Rao, Dominic Zeng Wang, Chi Hay Tong, and Ingmar Posner. V ote3deep: Fast ob- ject detection in 3d point clouds using efficient convolu- tional neural networks. In IEEE International Conference on Robotics and Automation, pages 1355–1361, 2017
2017
-
[23]
A point set generation network for 3d object reconstruction from a sin- gle image
Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3d object reconstruction from a sin- gle image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 605–613, 2017. 12
2017
-
[24]
Are we ready for autonomous driving? the kitti vision bench- mark suite
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision bench- mark suite. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3354– 3361, 2012
2012
-
[25]
Sub- manifold sparse convolutional networks
Benjamin Graham and Laurens Van der Maaten. Sub- manifold sparse convolutional networks. arXiv preprint arXiv:1706.01307, 2017
2017 arXiv
-
[26]
Scenenet: Understanding real world indoor scenes with synthetic data
Ankur Handa, Viorica Patraucean, Vijay Badrinarayanan, Si- mon Stent, and Roberto Cipolla. Scenenet: Understanding real world indoor scenes with synthetic data. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4077–4085, 2016
2016
-
[27]
Is your hd map constructor reliable under sensor corruptions? In Advances in Neural Informa- tion Processing Systems, pages 22441–22482, 2024
Xiaoshuai Hao, Mengchuan Wei, Yifan Yang, Haimei Zhao, Hui Zhang, Yi Zhou, Qiang Wang, Weiming Li, Lingdong Kong, and Jing Zhang. Is your hd map constructor reliable under sensor corruptions? In Advances in Neural Informa- tion Processing Systems, pages 22441–22482, 2024
2024
-
[28]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016
2016
-
[29]
Unified 3d and 4d panoptic segmentation via dynamic shifting networks
Fangzhou Hong, Lingdong Kong, Hui Zhou, Xinge Zhu, Hongsheng Li, and Ziwei Liu. Unified 3d and 4d panoptic segmentation via dynamic shifting networks. IEEE Transac- tions on Pattern Analysis and Machine Intelligence , 46(5): 3480–3495, 2024
2024
-
[30]
Squeeze-and-excitation networks
Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018
2018
-
[31]
Tri-perspective view for vision-based 3d semantic occupancy prediction
Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Tri-perspective view for vision-based 3d semantic occupancy prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9223–9232, 2023
2023
-
[32]
Sym- phonize 3d semantic scene completion with contextual in- stance queries
Haoyi Jiang, Tianheng Cheng, Naiyu Gao, Haoyang Zhang, Tianwei Lin, Wenyu Liu, and Xinggang Wang. Sym- phonize 3d semantic scene completion with contextual in- stance queries. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20258– 20...
2024
-
[33]
Rethinking range view representation for lidar segmentation
Lingdong Kong, Youquan Liu, Runnan Chen, Yuexin Ma, Xinge Zhu, Yikang Li, Yuenan Hou, Yu Qiao, and Ziwei Liu. Rethinking range view representation for lidar segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 228–240, 2023
2023
-
[34]
Benchmarking 3d perception robustness to common corruptions and sensor failure
Lingdong Kong, Youquan Liu, Xin Li, Runnan Chen, Wen- wei Zhang, Jiawei Ren, Liang Pan, Kai Chen, and Ziwei Liu. Benchmarking 3d perception robustness to common corruptions and sensor failure. In International Conference on Learning Representations 2023 Workshop on Scene Rep- ...
2023
-
[35]
Robo3d: Towards robust and reliable 3d perception against corruptions
Lingdong Kong, Youquan Liu, Xin Li, Runnan Chen, Wen- wei Zhang, Jiawei Ren, Liang Pan, Kai Chen, and Ziwei Liu. Robo3d: Towards robust and reliable 3d perception against corruptions. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19994–20006, 2023
2023
-
[36]
Lasermix for semi-supervised lidar semantic segmentation
Lingdong Kong, Jiawei Ren, Liang Pan, and Ziwei Liu. Lasermix for semi-supervised lidar semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21705–21715, 2023
2023
-
[37]
Cottereau, and Wei Tsang Ooi
Lingdong Kong, Shaoyuan Xie, Hanjiang Hu, Lai Xing Ng, Benoit R. Cottereau, and Wei Tsang Ooi. Robodepth: Robust out-of-distribution depth estimation under corruptions. In Advances in Neural Information Processing Systems , pages 21298–21342, 2023
2023
-
[38]
Largead: Large-scale cross-sensor data pretraining for au- tonomous driving
Lingdong Kong, Xiang Xu, Youquan Liu, Jun Cen, Runnan Chen, Wenwei Zhang, Liang Pan, Kai Chen, and Ziwei Liu. Largead: Large-scale cross-sensor data pretraining for au- tonomous driving. arXiv preprint arXiv:2501.04005, 2025
2025
-
[39]
Multi- modal data-efficient 3d scene understanding for autonomous driving
Lingdong Kong, Xiang Xu, Jiawei Ren, Wenwei Zhang, Liang Pan, Kai Chen, Wei Tsang Ooi, and Ziwei Liu. Multi- modal data-efficient 3d scene understanding for autonomous driving. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(5):3748–3765, 2025
2025
-
[40]
Hierarchical temporal context learning for camera-based semantic scene comple- tion
Bohan Li, Jiajun Deng, Wenyao Zhang, Zhujin Liang, Da- long Du, Xin Jin, and Wenjun Zeng. Hierarchical temporal context learning for camera-based semantic scene comple- tion. In European Conference on Computer Vision , pages 131–148. Springer, 2024
2024
-
[41]
Brgscene: Bridging stereo geometry and bev representation with reliable mutual interaction for se- mantic scene completion
Bohan Li, Yasheng Sun, Zhujin Liang, Dalong Du, Zhuanghui Zhang, Xiaofeng Wang, Yunnan Wang, Xin Jin, and Wenjun Zeng. Brgscene: Bridging stereo geometry and bev representation with reliable mutual interaction for se- mantic scene completion. In Proceedings of the Interna- tio...
2024
-
[42]
Anisotropic convolutional networks for 3d semantic scene completion
Jie Li, Kai Han, Peng Wang, Yu Liu, and Xia Yuan. Anisotropic convolutional networks for 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3351– 3359, 2020
2020
-
[43]
Less is more: Reducing task and model complexity for 3d point cloud se- mantic segmentation
Li Li, Hubert PH Shum, and Toby P Breckon. Less is more: Reducing task and model complexity for 3d point cloud se- mantic segmentation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 9361–9371, 2023
2023
-
[44]
Li Li, Hubert PH Shum, and Toby P. Breckon. Rapid-seg: Range-aware pointwise distance distribution networks for 3d lidar segmentation. In European Conference on Computer Vision, pages 222–241. Springer, 2024
2024
-
[45]
Coarse3d: Class-prototypes for contrastive learning in weakly-supervised 3d point cloud segmentation
Rong Li, Anh-Quan Cao, and Raoul de Charette. Coarse3d: Class-prototypes for contrastive learning in weakly-supervised 3d point cloud segmentation. arXiv preprint arXiv:2210.01784, 2022
2022 arXiv
-
[46]
Seeground: See and ground for zero-shot open-vocabulary 3d visual grounding
Rong Li, Shijie Li, Lingdong Kong, Xulei Yang, and Jun- wei Liang. Seeground: See and ground for zero-shot open-vocabulary 3d visual grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3707–3717, 2025
2025
-
[47]
V oxformer: Sparse voxel transformer for camera- based 3d semantic scene completion
Yiming Li, Zhiding Yu, Christopher Choy, Chaowei Xiao, Jose M Alvarez, Sanja Fidler, Chen Feng, and Anima Anand- kumar. V oxformer: Sparse voxel transformer for camera- based 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...
2023
-
[48]
Is your lidar placement optimized for 3d scene understanding? In Advances in Neural Information Process- ing Systems, pages 34980–35017, 2024
Ye Li, Lingdong Kong, Hanjiang Hu, Xiaohao Xu, and Xi- aonan Huang. Is your lidar placement optimized for 3d scene understanding? In Advances in Neural Information Process- ing Systems, pages 34980–35017, 2024
2024
-
[49]
Sparse4d: Multi-view 3d object detec- tion with sparse spatial-temporal fusion
Xuewu Lin, Tianwei Lin, Zixiang Pei, Lichao Huang, and Zhizhong Su. Sparse4d: Multi-view 3d object detec- tion with sparse spatial-temporal fusion. arXiv preprint arXiv:2211.10581, 2022
2022 arXiv
-
[50]
See and think: Disentangling semantic scene completion
Shice Liu, Yu Hu, Yiming Zeng, Qiankun Tang, Beibei Jin, Yinhe Han, and Xiaowei Li. See and think: Disentangling semantic scene completion. In Proceedings of the Advances in Neural Information Processing Systems , pages 261–272, 2018
2018
-
[51]
Uniseg: A unified multi-modal li- dar segmentation network and the openpcseg codebase
Youquan Liu, Runnan Chen, Xin Li, Lingdong Kong, Yuchen Yang, Zhaoyang Xia, Yeqi Bai, Xinge Zhu, Yuexin Ma, Yikang Li, et al. Uniseg: A unified multi-modal li- dar segmentation network and the openpcseg codebase. In Proceedings of the IEEE/CVF International Conference on Compu...
2023
-
[52]
Segment any point cloud sequences by distilling vision foundation models
Youquan Liu, Lingdong Kong, Jun Cen, Runnan Chen, Wen- wei Zhang, Liang Pan, Kai Chen, and Ziwei Liu. Segment any point cloud sequences by distilling vision foundation models. In Advances in Neural Information Processing Sys- tems, pages 37193–37229, 2023
2023
-
[53]
Multi-space alignments towards universal lidar segmentation
Youquan Liu, Lingdong Kong, Xiaoyang Wu, Runnan Chen, Xin Li, Liang Pan, Ziwei Liu, and Yuexin Ma. Multi-space alignments towards universal lidar segmentation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14648–14661, 2024
2024
-
[54]
Self-supervised image- to-point distillation via semantically tolerant contrastive loss
Anas Mahmoud, Jordan SK Hu, Tianshu Kuai, Ali Harakeh, Liam Paull, and Steven L Waslander. Self-supervised image- to-point distillation via semantically tolerant contrastive loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7102–...
2023
-
[55]
Lidpm: Rethinking point diffusion for lidar scene completion
Tetiana Martyniuk, Gilles Puy, Alexandre Boulch, Re- naud Marlet, and Raoul de Charette. Lidpm: Rethinking point diffusion for lidar scene completion. arXiv preprint arXiv:2504.17791, 2025
2025 arXiv
-
[56]
Occupancy networks: Learning 3d reconstruction in function space
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Se- bastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 4460–4470, 2019
2019
-
[57]
Train till you drop: Towards stable and robust source-free unsupervised 3d do- main adaptation
Bj ¨orn Michele, Alexandre Boulch, Tuan-Hung Vu, Gilles Puy, Renaud Marlet, and Nicolas Courty. Train till you drop: Towards stable and robust source-free unsupervised 3d do- main adaptation. In European Conference on Computer Vi- sion, pages 1–19. Springer, 2024
2024
-
[58]
Tempo- ral consistent 3d lidar representation learning for semantic perception in autonomous driving
Lucas Nunes, Louis Wiesmann, Rodrigo Marcuzzi, Xieyuanli Chen, Jens Behley, and Cyrill Stachniss. Tempo- ral consistent 3d lidar representation learning for semantic perception in autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...
2023
-
[59]
Deepsdf: Learning con- tinuous signed distance functions for shape representation
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning con- tinuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 165–174, 2019
2019
-
[60]
Convolutional occupancy networks
Songyou Peng, Michael Niemeyer, Lars Mescheder, Marc Pollefeys, and Andreas Geiger. Convolutional occupancy networks. In European Conference on Computer Vision , pages 523–540. Springer, 2020
2020
-
[61]
Learning to adapt sam for segmenting cross-domain point clouds
Xidong Peng, Runnan Chen, Feng Qiao, Lingdong Kong, Youquan Liu, Yujing Sun, Tai Wang, Xinge Zhu, and Yuexin Ma. Learning to adapt sam for segmenting cross-domain point clouds. In European Conference on Computer Vision, pages 54–71. Springer, 2024
2024
-
[62]
Using a waffle iron for automotive point cloud semantic segmenta- tion
Gilles Puy, Alexandre Boulch, and Renaud Marlet. Using a waffle iron for automotive point cloud semantic segmenta- tion. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 3379–3389, 2023
2023
-
[63]
Three pillars improving vision foundation model distillation for lidar
Gilles Puy, Spyros Gidaris, Alexandre Boulch, Oriane Sim´eoni, Corentin Sautier, Patrick P´erez, Andrei Bursuc, and Renaud Marlet. Three pillars improving vision foundation model distillation for lidar. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...
2024
-
[64]
Completing 3d object shape from one depth image
Jason Rock, Tanmay Gupta, Justin Thorsen, JunYoung Gwak, Daeyun Shin, and Derek Hoiem. Completing 3d object shape from one depth image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2484–2493, 2015
2015
-
[65]
Lmscnet: Lightweight multiscale 3d se- mantic completion
Luis Roldao et al. Lmscnet: Lightweight multiscale 3d se- mantic completion. In International Conference on 3D Vi- sion, pages 111–119, 2020
2020
-
[66]
You never get a second chance to make a good first impression: Seeding ac- tive learning for 3d semantic segmentation
Nermin Samet, Oriane Sim ´eoni, Gilles Puy, Georgy Poni- matkin, Renaud Marlet, and Vincent Lepetit. You never get a second chance to make a good first impression: Seeding ac- tive learning for 3d semantic segmentation. In Proceedings of the IEEE/CVF International Conference o...
2023
-
[67]
Milan: Milli-annotations for lidar semantic segmen- tation
Nermin Samet, Gilles Puy, Oriane Sim ´eoni, and Renaud Marlet. Milan: Milli-annotations for lidar semantic segmen- tation. arXiv preprint arXiv:2407.15797, 2024
2024 arXiv
-
[68]
Image-to-lidar self-supervised distillation for autonomous driving data
Corentin Sautier, Gilles Puy, Spyros Gidaris, Alexandre Boulch, Andrei Bursuc, and Renaud Marlet. Image-to-lidar self-supervised distillation for autonomous driving data. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 9891–9901, 2022
2022
-
[69]
Bevcontrast: Self-supervision in bev space for automotive lidar point clouds
Corentin Sautier, Gilles Puy, Alexandre Boulch, Renaud Marlet, and Vincent Lepetit. Bevcontrast: Self-supervision in bev space for automotive lidar point clouds. In Interna- tional Conference on 3D Vision, pages 559–568, 2024
2024
-
[70]
Clustering is back: Reaching state-of-the-art lidar instance segmentation without training
Corentin Sautier, Gilles Puy, Alexandre Boulch, Renaud Marlet, and Vincent Lepetit. Clustering is back: Reaching state-of-the-art lidar instance segmentation without training. arXiv preprint arXiv:2503.13203, 2025
2025
-
[71]
Indoor segmentation and support inference from rgbd images
Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In European Conference on Computer Vision, pages 746–760. Springer, 2012
2012
-
[72]
Chang, Manolis Savva, and Thomas Funkhouser
Shuran Song, Fisher Yu, Andy Zeng, Angel X. Chang, Manolis Savva, and Thomas Funkhouser. Semantic scene 14 completion from a single depth image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1746–1754, 2017
2017
-
[73]
Searching efficient 3d archi- tectures with sparse point-voxel convolution
Haotian Tang, Zhijian Liu, Shengyu Zhao, Yujun Lin, Ji Lin, Hanrui Wang, and Song Han. Searching efficient 3d archi- tectures with sparse point-voxel convolution. In European Conference on Computer Vision , pages 685–702. Springer, 2020
2020
-
[74]
Semantic scene completion via integrating instances and scene in-the-loop
Jiaxiang Tang, Xiaokang Chen, Jingbo Wang, and Gang Zeng. Semantic scene completion via integrating instances and scene in-the-loop. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 324–333, 2021
2021
-
[75]
Meta- rangeseg: Lidar sequence semantic segmentation using mul- tiple feature aggregation
Song Wang, Jianke Zhu, and Ruixiang Zhang. Meta- rangeseg: Lidar sequence semantic segmentation using mul- tiple feature aggregation. IEEE Robotics and Automation Letters, 7(4):9739–9746, 2022
2022
-
[76]
Lidar2map: In defense of lidar-based semantic map construction using online camera distillation
Song Wang, Wentong Li, Wenyu Liu, Xiaolu Liu, and Jianke Zhu. Lidar2map: In defense of lidar-based semantic map construction using online camera distillation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5186–5195, 2023
2023
-
[77]
Not all voxels are equal: Hardness-aware semantic scene completion with self- distillation
Song Wang, Yu Jiawei, Li Wentong, Liu Wenyu, Liu Xi- aolu, Chen Junbo, and Zhu Jianke. Not all voxels are equal: Hardness-aware semantic scene completion with self- distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14792–...
2024
-
[78]
Reliocc: Towards reli- able semantic occupancy prediction via uncertainty learning
Song Wang, Zhongdao Wang, Jiawei Yu, Wentong Li, Bailan Feng, Junbo Chen, and Jianke Zhu. Reliocc: Towards reli- able semantic occupancy prediction via uncertainty learning. In arXiv:2409.18026, 2024
2024 arXiv
-
[79]
Pointlora: Low-rank adaptation with to- ken selection for point cloud learning
Song Wang, Xiaolu Liu, Lingdong Kong, Jianyun Xu, Chun- yong Hu, Gongfan Fang, Wentong Li, Jianke Zhu, and Xinchao Wang. Pointlora: Low-rank adaptation with to- ken selection for point cloud learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...
2025
-
[80]
Ffnet: Frequency fu- sion network for semantic scene completion
Xuzhi Wang, Di Lin, and Liang Wan. Ffnet: Frequency fu- sion network for semantic scene completion. In Proceed- ings of the AAAI Conference on Artificial Intelligence, pages 2550–2557, 2022
2022
-
[81]
Nuc-net: Non-uniform cylindrical partition network for ef- ficient lidar semantic segmentation
Xuzhi Wang, Wei Feng, Lingdong Kong, and Liang Wan. Nuc-net: Non-uniform cylindrical partition network for ef- ficient lidar semantic segmentation. IEEE Transactions on Circuits and Systems for Video Technology, 2025
2025
-
[82]
H2gformer: Horizontal-to-global voxel transformer for 3d semantic scene completion
Yu Wang and Chao Tong. H2gformer: Horizontal-to-global voxel transformer for 3d semantic scene completion. In Pro- ceedings of the AAAI Conference on Artificial Intelligence , pages 5722–5730, 2024
2024
-
[83]
Scpnet: Se- mantic scene completion on point cloud
Zhaoyang Xia, Youquan Liu, Xin Li, Xinge Xu, Yuexin Ma, Yikang Li, Yuenan Hou, and Yu Qiao. Scpnet: Se- mantic scene completion on point cloud. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17642–17651, 2023
2023
-
[84]
Are vlms ready for autonomous driving? an empiri- cal study from the reliability, data, and metric perspectives
Shaoyuan Xie, Lingdong Kong, Yuhao Dong, Chonghao Sima, Wenwei Zhang, Qi Alfred Chen, Ziwei Liu, and Liang Pan. Are vlms ready for autonomous driving? an empiri- cal study from the reliability, data, and metric perspectives. arXiv preprint arXiv:2501.04003, 2025
2025 arXiv
-
[85]
Benchmarking and im- proving bird’s eye view perception robustness in autonomous driving
Shaoyuan Xie, Lingdong Kong, Wenwei Zhang, Jiawei Ren, Liang Pan, Kai Chen, and Ziwei Liu. Benchmarking and im- proving bird’s eye view perception robustness in autonomous driving. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(5):3878–3894, 2025
2025
-
[86]
4d contrastive superflows are dense 3d representation learners
Xiang Xu, Lingdong Kong, Hui Shuai, Wenwei Zhang, Liang Pan, Kai Chen, Ziwei Liu, and Qingshan Liu. 4d contrastive superflows are dense 3d representation learners. In European Conference on Computer Vision, pages 58–80. Springer, 2024
2024
-
[87]
Frnet: Frustum-range networks for scalable lidar segmen- tation
Xiang Xu, Lingdong Kong, Hui Shuai, and Qingshan Liu. Frnet: Frustum-range networks for scalable lidar segmen- tation. IEEE Transactions on Image Processing , 34:2173– 2186, 2025
2025
-
[88]
Limoe: Mixture of lidar represen- tation learners from automotive scenes
Xiang Xu, Lingdong Kong, Hui Shuai, Liang Pan, Ziwei Liu, and Qingshan Liu. Limoe: Mixture of lidar represen- tation learners from automotive scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27368–27379, 2025
2025
-
[89]
Bi-ssc: Geometric-semantic bidirectional fusion for camera-based 3d semantic scene completion
Yujie Xue, Ruihui Li, Fan Wu, Zhuo Tang, Kenli Li, and Mingxing Duan. Bi-ssc: Geometric-semantic bidirectional fusion for camera-based 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 20124–20134, 2024
2024
-
[90]
Sparse single sweep lidar point cloud segmentation via learning contextual shape priors from scene completion
Xu Yan, Jiantao Gao, Jie Li, Ruimao Zhang, Zhen Li, Rui Huang, and Shuguang Cui. Sparse single sweep lidar point cloud segmentation via learning contextual shape priors from scene completion. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 3101–3109, 2021
2021
-
[91]
2dpass: 2d priors assisted semantic segmentation on lidar point clouds
Xu Yan, Jiantao Gao, Chaoda Zheng, Chao Zheng, Ruimao Zhang, Shuguang Cui, and Zhen Li. 2dpass: 2d priors assisted semantic segmentation on lidar point clouds. In European Conference on Computer Vision, pages 677–695. Springer, 2022
2022
-
[92]
Ndc-scene: Boost monocular 3d semantic scene completion in normalized de- vice coordinates space
Jiawei Yao, Chuming Li, Keqiang Sun, Yingjie Cai, Hao Li, Wanli Ouyang, and Hongsheng Li. Ndc-scene: Boost monocular 3d semantic scene completion in normalized de- vice coordinates space. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9421– 9...
2023
-
[93]
Efficient semantic scene comple- tion network with spatial group convolution
Jiahui Zhang, Hao Zhao, Anbang Yao, Yurong Chen, Li Zhang, and Hongen Liao. Efficient semantic scene comple- tion network with spatial group convolution. In European Conference on Computer Vision , pages 733–749. Springer, 2018
2018
-
[94]
Occformer: Dual-path transformer for vision-based 3d semantic occu- pancy prediction
Yunpeng Zhang, Zheng Zhu, and Dalong Du. Occformer: Dual-path transformer for vision-based 3d semantic occu- pancy prediction. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 9433–9443, 2023
2023
-
[95]
Monoocc: Digging into monocular semantic oc- cupancy prediction
Yupeng Zheng, Xiang Li, Pengfei Li, Yuhang Zheng, Bu Jin, Chengliang Zhong, Xiaoxiao Long, Hao Zhao, and Qichao Zhang. Monoocc: Digging into monocular semantic oc- cupancy prediction. In IEEE International Conference on Robotics and Automation, pages 18398–18405, 2024. 15
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.