Pith. sign in

REVIEW 3 major objections 5 minor 50 references

UP-Fuse: Uncertainty-guided LiDAR-Camera Fusion for 3D Panoptic Segmentation

T0 review · 3 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read UP-Fuse claims that learning per-pixel camera uncertainty from synthetic degradations lets LiDAR-camera fusion stay near its LiDAR-only floor when the camera fails or drifts.

desk verdict Solid fusion paper with a genuinely new uncertainty-gating idea and a useful if unvalidated Waymo benchmark, but the calibration-drift robustness claim is not actually backed by the proposed mechanism. read the letter →

arxiv 2602.19349 v2 pith:RGBVSFHM submitted 2026-02-22 cs.CV cs.AI

Rohit Mohan , Florian Drews , Yakov Miron , Daniele Cattaneo , Abhinav Valada This is my paper · ORCID
classification cs.CVcs.AI
keywords 3DpanopticsegmentationLiDAR-camerafusionuncertaintyestimationrange-viewrepresentationsensorfailurerobustnesscalibrationdriftdeformableattentionWaymo
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the reliability of a camera sensor can be learned and used to gate LiDAR-camera fusion, so that fusion-based 3D panoptic segmentation does not collapse when the camera degrades, drifts, or fails. UP-Fuse trains a small network to predict, at every range-view pixel, how much camera features change under synthetic image corruptions; that predicted instability becomes an uncertainty map that down-weights untrustworthy visual features before they are fused with LiDAR. The headline result is that with the camera completely removed at inference, the fused model drops only 1.2 panoptic-quality points, whereas the compared fusion methods drop 4.2-5.0 points and fall below their LiDAR-only baselines. Under 5 degrees of LiDAR-camera misalignment, the model loses 4.4% versus more than 8% for baselines, and it survives a daytime-to-nighttime shift without degradation. If correct, this matters because it offers a practical, architecture-level answer to when a perception system should trust each sensor in safety-critical driving.

What carries the argument

The central mechanism is the uncertainty-guided fusion gate. A 3-layer MLP is trained with a Huber loss to regress the L2 distance between range-view camera features of an original image and a corrupted counterpart; the predicted distance is converted to a score U = 1 - exp(-d) in [0,1]. Before deformable cross-modal attention, camera features are multiplied by (1 - U), so high-uncertainty pixels are attenuated. This is paired with a Hybrid 2D-3D Panoptic Decoder that gathers K mask features from pixels whose stored range is nearest to each point's true range, producing per-point mask logits and avoiding the label-bleeding and 360-degree wrap-around problems of pure 2D range-view decoding.

What would settle it

Feed the trained model a held-out set degraded by a failure mode outside the training augmentation pool—e.g., column-wise sensor readout faults or mud splatter over half the lens—and measure both the predicted uncertainty maps and the PQ gap to the LiDAR-only variant. If the gate stays low in corrupted regions while the PQ drop exceeds the roughly 1.2 points observed under full dropout, the synthetic-degradation proxy in Eq. (5)-(11) is falsified. A sharper test: correlate predicted per-pixel uncertainty with an independent geometric misalignment measure, such as reprojection error under known

Watch

Extended reading notes

Core claim

The paper's central claim is that reliability, not just relevance, can be learned for cross-modal fusion. UP-Fuse trains a small MLP to predict, per range-view pixel, the L2 distance between camera features from an original image and a corrupted copy; that predicted instability (converted to a gate score in [0,1]) multiplies camera features before deformable attention fuses them with LiDAR. Reported payoff: removing the camera at inference costs 1.2 PQ points (versus 4.2-5.0 for baselines), 5-degree extrinsic rotation costs 4.4% (versus over 8%), and daytime-to-nighttime shift causes no drop. A hybrid 2D-3D decoder lifts fused features to point-cloud masks, and a new Panoptic Waymo benchmark

Load-bearing premise

The load-bearing premise is that synthetic degradations applied during training make camera features unstable in the same way that real camera failure, misalignment, or night exposure does at deployment; if a real failure mode produces feature changes unlike the training augmentations, the learned uncertainty gate will not recognize it and will not down-weight the camera.

Editorial extensions

If this is right

  • Removing the camera at inference costs UP-Fuse only 1.2 PQ points relative to its LiDAR-only variant, while the compared fusion methods fall 4.2-5.0 points below their own LiDAR-only baselines.
  • Under LiDAR-camera misalignment up to 5 degrees, UP-Fuse loses 4.4% PQ rather than the 8%+ lost by the compared methods, because the gate progressively suppresses misaligned visual features.
  • On a daytime-trained model evaluated on nighttime scenes, UP-Fuse holds its performance (a 0.1 PQ gain), while the baselines drop 2.1-3.1 points.
  • Because all computations happen in a shared 2D range-view space, the method runs at 5.7 FPS on Panoptic nuScenes, about 6x faster than the strongest fusion baseline.
  • The paper's stated limitation: under severe misalignment the model suppresses the camera rather than correcting it, so performance is bounded by the LiDAR-only baseline; full fusion gains return only with explicit extrinsic refinement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The uncertainty head is supervised only by feature divergence, not by panoptic labels, which suggests the same learned gate could transfer to other LiDAR-camera tasks such as 3D object detection without relabeling — but the paper does not test this.
  • Because the augmentation pool determines what counts as 'unreliable,' the method's safety margin depends on covering the real failure modes of the target sensor; an obvious test is adding failures outside the pool (e.g., partial column readout faults, lens mud) and checking whether PQ drop stays near 1.2.
  • With a 1.2-point worst-case gap to LiDAR-only under camera failure and a nominal fusion gain of 5.8 points, a deployment policy could decide when to trust the camera branch and when to run LiDAR-only; UP-Fuse provides the per-pixel signal such a policy would need, though the paper does not build the policy.
  • The frozen camera encoder plus lightweight uncertainty MLP raises the question whether the uncertainty head alone transfers across sensor rigs or datasets, which could be tested by fine-tuning only the gate on a new rig.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents UP-Fuse, a range-view LiDAR-camera fusion architecture for 3D panoptic segmentation. Camera features are projected into the LiDAR range-view and fused with LiDAR features through uncertainty-modulated deformable cross-modal attention. The uncertainty head is trained to regress the L2 divergence between features extracted from a clean image and a synthetically corrupted counterpart (Eq. 5), and the predicted divergence is converted into a gating signal (Eqs. 9–11). A hybrid 2D-3D decoder predicts per-point panoptic masks. The authors report strong results on Panoptic nuScenes and SemanticKITTI, introduce a new Panoptic Waymo benchmark, and claim robustness to camera sensor dropout, calibration drift, and visual domain shift. The paper includes detailed ablations, a new benchmark, and makes code/models publicly available.

Significance. If the robustness claims are fully supported, the work addresses a practically important failure mode in multi-modal perception: knowing when to trust the camera stream. The efficiency gain (5.7 FPS vs. 0.9 FPS for IAL) is a concrete practical contribution, and the new Panoptic Waymo benchmark may be useful to the community. The ablations are systematic and the uncertainty-guided gating mechanism is an interesting design. However, the load-bearing robustness claims are only partially secured: the mechanism as described has no way to detect geometric misalignment, and the nighttime evaluation partially overlaps with the training augmentation pool. With additional experiments or a modified mechanism, the central idea could be made convincing.

major comments (3)
  1. [§III-B, Eqs. (5)–(11); §IV-D, Fig. 4] The calibration-drift robustness result is not explained by the proposed mechanism. The uncertainty target in Eq. (5) is the L2 divergence between features of the original and a photometrically/structurally corrupted image; the augmentation list in Sec. VI contains only non-spatial corruptions. The uncertainty head Uθ,s in Eq. (6) consumes only the RV-aligned camera features FC,s. Under an extrinsics rotation, the image itself is clean and locally well-formed, but the projected features are spatially displaced. Nothing in Uθ,s's training teaches it to raise the uncertainty for geometrically misaligned features, and U has no access to LiDAR-camera agreement. The Fig. 4 margin (4.4% PQ drop vs. >8% for baselines) could therefore stem from deformable-attention tolerance or from the LiDAR-only floor rather than from uncertainty gating. The paper needs either (a) to make U depend on a cross-m
  2. [Tables I, IV; Fig. 4; Table V] All quantitative results are reported from single runs, with no error bars or multiple seeds. The main head-to-head advantages over the IAL baseline are 0.4 PQ on Panoptic nuScenes (Table I, 80.7 vs. 80.3) and 0.5 PQ on Panoptic Waymo (Table IV, 60.9 vs. 60.4). These margins are small enough that run-to-run variation could reverse them. Even the larger robustness margins in Table V and Fig. 4 need confidence intervals to support the claimed separation. Please report mean ± std over at least three seeds for the central comparisons, or otherwise justify why single runs are reliable.
  3. [§VI-C and Table VI] The nighttime 'visual domain shift' evaluation is partially in-distribution. The uncertainty training explicitly includes histogram matching against Dark Zurich (Sec. VI-C), so the model has been trained on photometric statistics resembling nighttime images. The claim that UP-Fuse is robust to a 'naturally occurring domain shift' is therefore not fully established; the result may reflect the model's exposure to the augmentation rather than a general mechanism. Please evaluate on a held-out degradation not present in the augmentation pool (e.g., a different night dataset or a separately defined corruption), or ablate Dark Zurich histogram matching to show that the robustness does not depend on this specific training augmentation.
minor comments (5)
  1. [Eq. (7) and Sec. VI] The Huber threshold δ=1.0 and the 0.5 probability of using I_aug = I_orig are free hyperparameters. A short sensitivity analysis would strengthen the paper.
  2. [Sec. IV-D, Table V] When the camera is removed at inference (L*), it is not specified whether the model receives a zero image, missing camera features, or a dropped view. Please state the exact protocol, since this affects the interpretation of the uncertainty mechanism.
  3. [Sec. VI, Fig. 3] The uncertainty heatmaps in Fig. 3 are shown for corruption and dropout, but not for calibration drift. Adding a drift heatmap would directly address the mechanism gap raised above.
  4. [Sec. IV-A, Panoptic Waymo] The proposed benchmark is built by merging semantic labels and 3D boxes. A short analysis of annotation quality (e.g., agreement with manual inspection or per-class consistency statistics) would help establish the reliability of the benchmark for training and evaluation.
  5. [Table VI] The note that the night split lacks four thing classes (bus, construction vehicle, trailer, traffic cone) should be reflected in the metric reporting; please clarify whether PQ is computed over the intersection of classes present in both day and night or over all classes with absent ones treated specially.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: uncertainty model is a trained auxiliary predictor; robustness evaluations are controlled; self-citations are not load-bearing.

full rationale

The derivation is not circular. Eq. (5) defines the uncertainty supervision as the L2 distance between frozen-encoder features of an original image and a corrupted counterpart; Eqs. (6)-(9) train a small MLP to regress that distance from the corrupted features, and Eq. (11) uses the resulting scalar to gate camera features. This is an auxiliary self-supervised task, not a restatement of the final panoptic objective. The panoptic loss (14) is independent, and the gating is optimized through the fused features, so no 'prediction' is identical to its input by construction. The sensor-dropout robustness result does overlap with a training augmentation ('Random Dropout: Replaces the full image with zeros'), but the paper retrains all fusion baselines from scratch with the same camera dropout augmentations (Sec. IV-D), so the comparison isolates the uncertainty mechanism rather than the augmentation. The visual-domain-shift evaluation also uses Dark Zurich histogram matching during training, limiting extrapolation, but again this is a training/evaluation overlap, not a circular derivation. The calibration-drift robustness claim is less well supported: the uncertainty loss (5) uses only non-spatial photometric corruptions and U_{theta,s} consumes only camera features, so nothing in the training signal explicitly teaches it to detect wrong extrinsics; the paper's own Limitations section concedes it only suppresses the visual stream and is bounded by the LiDAR-only baseline under severe misalignment. This is a mechanism/evidence gap, not circularity. Self-citations (e.g., [5], [7], [27]-[29]) are used for dataset protocol, range-view assignment, and histogram matching, but none carries the central fusion claim or forbids alternatives. Therefore no circular step is exhibited.

Assumptions & free parameters 8 free parameters · 6 assumptions · 0 invented entities

The central robustness mechanism is supported by an auxiliary uncertainty loss whose targets are defined internally (features of original vs. synthetically corrupted images). The main load-bearing free choices are the augmentation pool, the instability-to-uncertainty mapping, and the decoder hyperparameters. The paper does not derive any of these from first principles; they are engineering choices validated on fixed benchmarks.

free parameters (8)
  • Huber threshold δ for uncertainty loss = 1.0
    Chosen by hand in Eq. (8); determines the robustness of the uncertainty regression and the magnitude of predicted instability.
  • Probability of no augmentation in uncertainty training = 0.5
    Sets the fraction of zero-uncertainty reference samples; directly shapes the uncertainty targets.
  • Uncertainty gating transform = U = 1 - exp(-d_pred)
    Hand-chosen monotone map from predicted instability to [0,1]; controls how strongly camera features are attenuated in Eq. (11).
  • Deformable attention sampling points P = 4
    Chosen for fusion attention in Eq. (10); affects how many visual samples are used per LiDAR query.
  • 3D-aware mask head neighborhood size K = 5
    Ablated in Fig. 5b; chosen on validation PQ as the best balance of context aggregation and feature specificity.
  • Number of object queries Nq = 300
    Standard Mask2Former setting; determines decoder capacity and the number of predicted masks.
  • Panoptic loss weights (λ_cls, λ_dice, λ_mask) = (5,5,100) for nuScenes; (2,5,50) for Waymo/SemanticKITTI
    Hand-tuned per dataset in Sec. IV-B; balances classification, mask, and Dice losses.
  • Uncertainty loss weight λ_unc = 1 (nuScenes) or 1 (Waymo/SemanticKITTI)
    Set manually in Sec. IV-B; controls the contribution of the uncertainty regression to total training.
assumptions (6)
  • domain assumption Aleatoric unreliability of camera features can be represented as the L2 distance between features of original and corrupted images
    This is the entire supervision signal for the uncertainty head in Eq. (5); if false, predicted uncertainty maps do not reflect true reliability under novel failure modes. No independent evidence is provided beyond the experiments.
  • domain assumption Dense depth completion [12] from sparse LiDAR projections provides sufficiently accurate camera-to-range-view feature alignment
    View transformation in Eq. (4) relies on this; systematic completion errors would misalign camera and LiDAR features before fusion.
  • domain assumption The range-view nearest-point projection preserves enough information for 3D panoptic segmentation
    Used in Sec. III-A1 and III-C; the 3D-aware decoder is designed to compensate for occlusion/ambiguity, but the initial representation is still a single range value per pixel.
  • ad hoc to paper Panoptic Waymo annotations obtained by merging semantic labels and 3D boxes are correct enough for training and evaluation
    No manual verification is reported; instances with fewer than 50 points are excluded and the motorcycle class is removed, so the benchmark is an automatically derived approximation.
  • domain assumption Random dropout (zero image) is a faithful model of real camera sensor failure at inference
    The L* robustness evaluation feeds zero images, and the same corruption appears in the training augmentation pool; real sensor failures may look different.
  • standard math Deformable attention, Mask2Former-style set prediction, and bipartite matching provide a sound formulation for panoptic mask prediction
    Adopted from [50,3]; the paper does not derive these but relies on their established behavior for the decoder.

how reviews work

0 comments
Cite this review

Pith. "Pith review of UP-Fuse: Uncertainty-guided LiDAR-Camera Fusion for 3D Panoptic Segmentation." pith.science (2026). https://pith.science/paper/RGBVSFHM

@misc{pith2026260219349,
  author       = {Pith},
  title        = {Pith review of: UP-Fuse: Uncertainty-guided LiDAR-Camera Fusion for 3D Panoptic Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RGBVSFHM}},
  note         = {Machine review of arXiv:2602.19349}
}
read the original abstract

LiDAR-camera fusion enhances 3D panoptic segmentation by leveraging camera images to complement sparse LiDAR scans, but it also introduces a critical failure mode. Under adverse conditions, degradation or failure of the camera sensor can significantly compromise the reliability of the perception system. To address this problem, we introduce UP-Fuse, a novel uncertainty-aware fusion framework in the 2D range-view that remains robust under camera sensor degradation, calibration drift, and sensor failure. Raw LiDAR data is first projected into the range-view and encoded by a LiDAR encoder, while camera features are simultaneously extracted and projected into the same shared space. At its core, UP-Fuse employs an uncertainty-guided fusion module that dynamically modulates cross-modal interaction using predicted uncertainty maps. These maps are learned by quantifying representational divergence under diverse visual degradations, ensuring that only reliable visual cues influence the fused representation. The fused range-view features are decoded by a novel hybrid 2D-3D transformer that mitigates spatial ambiguities inherent to the 2D projection and directly predicts 3D panoptic segmentation masks. Extensive experiments on Panoptic nuScenes, SemanticKITTI, and our introduced Panoptic Waymo benchmark demonstrate the efficacy and robustness of UP-Fuse, which maintains strong performance even under severe visual corruption or misalignment, making it well suited for robotic perception in safety-critical settings.

Figures

Figures reproduced from arXiv: 2602.19349 by the authors.

Figure 1
Figure 1. Visualization of 3D panoptic segmentation: green indicates correct and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the proposed UP-Fuse architecture. LiDAR and multi-view camera images are fused onto a shared space of range-view feature representations. The Uncertainty-Aware Fusion Module adaptively integrates modalities via uncertainty-weighted deformable cross-modal interaction to attenuate unreliable visual cues. Finally, a Hybrid 2D-3D Panoptic Decoder generates 3D predictions. Paths and blocks shown in brown… view at source ↗
Figure 3
Figure 3. Illustration of our uncertainty module on a Panoptic nuScenes sample. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Robust performance comparison on the Panoptic nuScenes validation [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Ablation studies on the key hyperparameters of our Hybrid 2D-3D [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Qualitative robustness comparison of 3D panoptic segmentation between [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Visualization of LiDAR-to-camera projection shifts under increasing calibration drift (rotation magnitude) from [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Dataset statistics of Panoptic Waymo. VII. STATISTICS OF THE PANOPTIC WAYMO DATASET The Waymo Open Dataset [39] contains 5.2B annotated LiDAR points across 22 semantic classes, offering more than five times the annotated LiDAR points of Panoptic nuScenes [5]. In additi…
Figure 9
Figure 9. Figure 9: Examples from the Panoptic Waymo dataset showing the multi-view camera images, corresponding LiDAR scan, and their panoptic ground truth. For [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Qualitative 3D Panoptic Segmentation results of our proposed UP-Fuse network versus the baseline LCPS architecture, on the Panoptic Waymo val set. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Visualization of 3D panoptic segmentation predictions of UP-Fuse on real-world scenes. A model trained on the Panoptic Waymo dataset is deployed [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Our in-house autonomous driving vehicle used to demonstrate the [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 6 linked inside Pith

  1. [1]

    Semantickitti: A dataset for semantic scene understanding of lidar sequences

    Jens Behley, Martin Garbade, Andres Milioto, Jan Quen- zel, Sven Behnke, Cyrill Stachniss, and Jurgen Gall. Semantickitti: A dataset for semantic scene understanding of lidar sequences. InProceedings of the IEEE/CVF international conference on computer vision, pages 9297– 9307, 2019

    2019

  2. [2]

    Weight uncertainty in neural network

    Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. InInternational conference on machine learning, pages 1613–1622, 2015

    2015

  3. [3]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022

    2022

  4. [4]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3213–3223, 2016

    2016

  5. [5]

    Panoptic nuscenes: A large-scale benchmark for lidar panoptic segmentation and tracking.IEEE Robotics and Automation Letters, 7(2):3795–3802, 2022

    Whye Kit Fong, Rohit Mohan, Juana Valeria Hurtado, Lubing Zhou, Holger Caesar, Oscar Beijbom, and Abhinav Valada. Panoptic nuscenes: A large-scale benchmark for lidar panoptic segmentation and tracking.IEEE Robotics and Automation Letters, 7(2):3795–3802, 2022

    2022

  6. [6]

    Maskrange: A mask-classification model for range-view based lidar segmentation.arXiv preprint arXiv:2206.12073, 2022

    Yi Gu, Yuming Huang, Chengzhong Xu, and Hui Kong. Maskrange: A mask-classification model for range-view based lidar segmentation.arXiv preprint arXiv:2206.12073, 2022

    arXiv 2022

  7. [7]

    Label- efficient lidar semantic segmentation with 2d-3d vision transformer adapters.arXiv preprint arXiv:2503.03299, 2025

    Julia Hindel, Rohit Mohan, Jelena Bratulic, Daniele Cattaneo, Thomas Brox, and Abhinav Valada. Label- efficient lidar semantic segmentation with 2d-3d vision transformer adapters.arXiv preprint arXiv:2503.03299, 2025

    arXiv 2025

  8. [8]

    Lidar-based panoptic segmentation via dynamic shifting network

    Fangzhou Hong, Hui Zhou, Xinge Zhu, Hongsheng Li, and Ziwei Liu. Lidar-based panoptic segmentation via dynamic shifting network. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13090–13099, 2021

    2021

Show all 50 references
  1. [9]

    What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017

    Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017

    2017

  2. [10]

    Panoptic segmentation

    Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Doll ´ar. Panoptic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9404–9413, 2019

    2019

  3. [11]

    Challenges in autonomous vehicle testing and validation.SAE Inter- national Journal of Transportation Safety, 4(1):15–24, 2016

    Philip Koopman and Michael Wagner. Challenges in autonomous vehicle testing and validation.SAE Inter- national Journal of Transportation Safety, 4(1):15–24, 2016

    2016

  4. [12]

    In defense of classical image processing: Fast depth com- pletion on the cpu

    Jason Ku, Ali Harakeh, and Steven L Waslander. In defense of classical image processing: Fast depth com- pletion on the cpu. In15th conference on computer and robot vision (CRV), pages 16–22, 2018

    2018

  5. [13]

    Simple and scalable predictive uncertainty estimation using deep ensembles.Advances in neural information processing systems, 30, 2017

    Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles.Advances in neural information processing systems, 30, 2017

    2017

  6. [14]

    Cpseg: Cluster-free panoptic segmentation of 3d lidar point clouds.arXiv preprint arXiv:2111.01723, 2021

    Enxu Li, Ryan Razani, Yixuan Xu, and Bingbing Liu. Cpseg: Cluster-free panoptic segmentation of 3d lidar point clouds.arXiv preprint arXiv:2111.01723, 2021

    2021 arXiv

  7. [15]

    Panoptic-phnet: Towards real- time and high-precision lidar panoptic segmentation via clustering pseudo heatmap

    Jinke Li, Xiao He, Yang Wen, Yuan Gao, Xiaoqiang Cheng, and Dan Zhang. Panoptic-phnet: Towards real- time and high-precision lidar panoptic segmentation via clustering pseudo heatmap. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11...

    2022

  8. [16]

    Center focusing network for real-time lidar panoptic segmentation

    Xiaoyan Li, Gang Zhang, Boyue Wang, Yongli Hu, and Baocai Yin. Center focusing network for real-time lidar panoptic segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13425–13434, 2023

    2023

  9. [17]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InEuropean conference on computer vision, pages 740–755, 2014

    2014

  10. [18]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021

    2021

  11. [19]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  12. [20]

    Amodal optical flow

    Maximilian Luz, Rohit Mohan, Ahmed Rida Sekkat, Oliver Sawade, Elmar Matthes, Thomas Brox, and Abhi- nav Valada. Amodal optical flow. InIEEE International Conference on Robotics and Automation (ICRA), pages 14677–14684, 2024

    2024

  13. [21]

    Mask-based panoptic lidar segmentation for autonomous driving.IEEE Robotics and Automation Letters, 8(2):1141–1148, 2023

    Rodrigo Marcuzzi, Lucas Nunes, Louis Wiesmann, Jens Behley, and Cyrill Stachniss. Mask-based panoptic lidar segmentation for autonomous driving.IEEE Robotics and Automation Letters, 8(2):1141–1148, 2023

    2023

  14. [22]

    Centerlps: Segment instances by centers for lidar panoptic segmentation

    Jianbiao Mei, Yu Yang, Mengmeng Wang, Zizhang Li, Xiaojun Hou, Jongwon Ra, Laijian Li, and Yong Liu. Centerlps: Segment instances by centers for lidar panoptic segmentation. InProceedings of the 31st ACM International Conference on Multimedia, pages 1884– 1894, 2023

    2023

  15. [23]

    Rangenet++: Fast and accurate lidar semantic segmentation

    Andres Milioto, Ignacio Vizzo, Jens Behley, and Cyrill Stachniss. Rangenet++: Fast and accurate lidar semantic segmentation. InIEEE/RSJ international conference on intelligent robots and systems, pages 4213–4220, 2019

    2019

  16. [24]

    Lidar panoptic segmentation for autonomous driving

    Andres Milioto, Jens Behley, Chris McCool, and Cyrill Stachniss. Lidar panoptic segmentation for autonomous driving. InIEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8505–8512, 2020

    2020

  17. [25]

    Forecastocc: Vision-based semantic occupancy forecasting.arXiv preprint arXiv:2602.08006, 2026

    Riya Mohan, Juana Valeria Hurtado, Rohit Mohan, and Abhinav Valada. Forecastocc: Vision-based semantic occupancy forecasting.arXiv preprint arXiv:2602.08006, 2026

    2026

  18. [26]

    Perceiving the invisible: Proposal-free amodal panoptic segmentation

    Rohit Mohan and Abhinav Valada. Perceiving the invisible: Proposal-free amodal panoptic segmentation. IEEE Robotics and Automation Letters, 7(4), 2022

    2022

  19. [27]

    Syn-mediverse: A multimodal synthetic dataset for intelligent scene understanding of healthcare facilities.IEEE Robotics and Automation Letters, 9(8):7094–7101, 2024

    Rohit Mohan, Jos ´e Arce, Sassan Mokhtar, Daniele Catta- neo, and Abhinav Valada. Syn-mediverse: A multimodal synthetic dataset for intelligent scene understanding of healthcare facilities.IEEE Robotics and Automation Letters, 9(8):7094–7101, 2024

    2024

  20. [28]

    Progressive multi-modal fusion for robust 3d object detection

    Rohit Mohan, Daniele Cattaneo, Florian Drews, and Abhinav Valada. Progressive multi-modal fusion for robust 3d object detection. In8th Annual Conference on Robot Learning, 2024

    2024

  21. [29]

    Panoptic out-of- distribution segmentation.IEEE Robotics and Automation Letters, 9(5):4075–4082, 2024

    Rohit Mohan, Kiran Kumaraswamy, Juana Valeria Hur- tado, K ¨ursat Petek, and Abhinav Valada. Panoptic out-of- distribution segmentation.IEEE Robotics and Automation Letters, 9(5):4075–4082, 2024

    2024

  22. [30]

    Open- set lidar panoptic segmentation guided by uncertainty- aware learning

    Rohit Mohan, Julia Hindel, Florian Drews, Claudius Gl¨aser, Daniele Cattaneo, and Abhinav Valada. Open- set lidar panoptic segmentation guided by uncertainty- aware learning. InIEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2224–2231, 2025

    2025

  23. [31]

    3d scene segmentation

    Slavcho Neshev, Krasimir Tonchev, Agata Manolova, and Vladimir Poulkov. 3d scene segmentation. a comprehen- sive survey and open problems.IEEE Access, 2025

    2025

  24. [32]

    How do images align and complement lidar? towards a harmonized multi-modal 3d panoptic segmentation.arXiv preprint arXiv:2505.18956, 2025

    Yining Pan, Qiongjie Cui, Xulei Yang, and Na Zhao. How do images align and complement lidar? towards a harmonized multi-modal 3d panoptic segmentation.arXiv preprint arXiv:2505.18956, 2025

    2025 arXiv

  25. [33]

    Gp-s3net: Graph-based panoptic sparse semantic segmentation network

    Ryan Razani, Ran Cheng, Enxu Li, Ehsan Taghavi, Yuan Ren, and Liu Bingbing. Gp-s3net: Graph-based panoptic sparse semantic segmentation network. InProceedings of the IEEE/CVF international conference on computer vision, pages 16076–16085, 2021

    2021

  26. [34]

    Guided curriculum model adaptation and uncertainty- aware evaluation for semantic nighttime image segmen- tation

    Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Guided curriculum model adaptation and uncertainty- aware evaluation for semantic nighttime image segmen- tation. InProceedings of the IEEE/CVF international conference on computer vision, pages 7374–7383, 2019

    2019

  27. [35]

    Bevcar: Camera-radar fusion for bev map and object segmentation

    Jonas Schramm, Niclas V ¨odisch, K ¨ursat Petek, B Ravi Kiran, Senthil Yogamani, Wolfram Burgard, and Abhinav Valada. Bevcar: Camera-radar fusion for bev map and object segmentation. InIEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1435–1442, 2024

    2024

  28. [36]

    Efficientlps: Efficient lidar panoptic segmentation.IEEE Transactions on Robotics, 38(3):1894–1914, 2021

    Kshitij Sirohi, Rohit Mohan, Daniel B ¨uscher, Wolfram Burgard, and Abhinav Valada. Efficientlps: Efficient lidar panoptic segmentation.IEEE Transactions on Robotics, 38(3):1894–1914, 2021

    1914

  29. [37]

    Panoptic-fusionnet: Camera-lidar fusion-based point cloud panoptic segmentation for autonomous driving

    Hamin Song, Jieun Cho, Jinsu Ha, Jaehyun Park, and Kichun Jo. Panoptic-fusionnet: Camera-lidar fusion-based point cloud panoptic segmentation for autonomous driving. Expert Systems with Applications, 251:123950, 2024

    2024

  30. [38]

    Pups: Point cloud unified panoptic segmentation

    Shihao Su, Jianyun Xu, Huanyu Wang, Zhenwei Miao, Xin Zhan, Dayang Hao, and Xi Li. Pups: Point cloud unified panoptic segmentation. InProceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 2339–2347, 2023

    2023

  31. [39]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. InProceedings of the IEEE/CVF conference on computer vis...

    2020

  32. [40]

    Kpconv: Flexible and deformable convolution for point clouds

    Hugues Thomas, Charles R Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, Fran c ¸ois Goulette, and Leonidas J Guibas. Kpconv: Flexible and deformable convolution for point clouds. InProceedings of the IEEE/CVF international conference on computer vision, pages 6411– 6420, 2019

    2019

  33. [41]

    Convoluted mixture of deep experts for robust semantic segmentation

    Abhinav Valada, Ankit Dhall, and Wolfram Burgard. Convoluted mixture of deep experts for robust semantic segmentation. InIEEE/RSJ International conference on intelligent robots and systems (IROS) workshop, state estimation and terrain perception for all terrain mobile robots, ...

    2016

  34. [42]

    Towards robust semantic segmentation using deep fusion

    Abhinav Valada, Gabriel Oliveira, Thomas Brox, and Wolfram Burgard. Towards robust semantic segmentation using deep fusion. InRobotics: Science and systems (RSS 2016) workshop, are the sceptics right? Limits and potentials of deep learning in robotics, volume 114, 2016

    2016

  35. [43]

    Position-guided point cloud panoptic segmentation transformer.International Journal of Computer Vision, 133(1):275–290, 2025

    Zeqi Xiao, Wenwei Zhang, Tai Wang, Chen Change Loy, Dahua Lin, and Jiangmiao Pang. Position-guided point cloud panoptic segmentation transformer.International Journal of Computer Vision, 133(1):275–290, 2025

    2025

  36. [44]

    Sparse cross-scale attention network for efficient lidar panoptic segmentation

    Shuangjie Xu, Rui Wan, Maosheng Ye, Xiaoyi Zou, and Tongyi Cao. Sparse cross-scale attention network for efficient lidar panoptic segmentation. InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 2920–2928, 2022

    2022

  37. [45]

    Aop-net: All-in-one perception network for lidar- based joint 3d object detection and panoptic segmentation

    Yixuan Xu, Hamidreza Fazlali, Yuan Ren, and Bingbing Liu. Aop-net: All-in-one perception network for lidar- based joint 3d object detection and panoptic segmentation. InIEEE Intelligent Vehicles Symposium (IV), pages 1–7, 2023

    2023

  38. [46]

    Cross modal transformer: Towards fast and robust 3d object detection

    Junjie Yan, Yingfei Liu, Jianjian Sun, Fan Jia, Shuailin Li, Tiancai Wang, and Xiangyu Zhang. Cross modal transformer: Towards fast and robust 3d object detection. InProceedings of the IEEE/CVF international conference on computer vision, pages 18268–18278, 2023

    2023

  39. [47]

    Lidar- multinet: Towards a unified multi-task network for lidar perception

    Dongqiangzi Ye, Zixiang Zhou, Weijia Chen, Yufei Xie, Yu Wang, Panqu Wang, and Hassan Foroosh. Lidar- multinet: Towards a unified multi-task network for lidar perception. InProceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 3231–3240, 2023

    2023

  40. [48]

    Lidar-camera panoptic segmentation via geometry-consistent and semantic-aware alignment

    Zhiwei Zhang, Zhizhong Zhang, Qian Yu, Ran Yi, Yuan Xie, and Lizhuang Ma. Lidar-camera panoptic segmentation via geometry-consistent and semantic-aware alignment. InProceedings of the IEEE/CVF international conference on computer vision, pages 3662–3671, 2023

    2023

  41. [49]

    Panoptic-polarnet: Proposal-free lidar point cloud panoptic segmentation

    Zixiang Zhou, Yang Zhang, and Hassan Foroosh. Panoptic-polarnet: Proposal-free lidar point cloud panoptic segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13194– 13203, 2021

    2021

  42. [50]

    Deformable detr: Deformable transformers for end-to-end object detection.arXiv preprint arXiv:2010.04159, 2020

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection.arXiv preprint arXiv:2010.04159, 2020. UP-Fuse: Uncertainty-guided LiDAR-Camera Fusion for 3D Panoptic Segmentation Rohit Mohan 1, F...

    2010 arXiv

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.