Pith. sign in

REVIEW 4 major objections 5 minor 99 references

Event-aided Semantic Scene Completion

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Pith's one-line read Fusing event data into the 2D-to-3D lifting step makes camera-based semantic scene completion more accurate and robust under degraded imaging.

desk verdict New dataset and a sensible fusion idea, but the real-world legs are weak and the headline gain is a single cell. read the letter →

arxiv 2502.02334 v1 pith:XIDGXH3Y submitted 2025-02-04 cs.CV cs.ROeess.IV

classification cs.CVcs.ROeess.IV
keywords semanticscenecompletioneventcameraRGB-eventfusion3DoccupancypredictionviewtransformationautonomousdrivingrobustnesstocorruptionDSEC-SSCbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that event cameras, which report brightness changes asynchronously, can make camera-based semantic scene completion—predicting a full 3D occupancy grid with semantic labels from images—more accurate and much more robust when ordinary images degrade. To test this, it builds DSEC-SSC, a real-world benchmark with event, RGB, and LiDAR data, and proposes EvSSC, a fusion framework whose Event-aided Lifting Module (ELM) mixes RGB and event features during the 2D-to-3D view-transformation step. Across transformer-based and LSS-based architectures, EvSSC improves mean IoU on clean benchmarks and, on corrupted SemanticKITTI-C, consistently beats RGB-only baselines under motion blur, fog, brightness, darkness, and shot noise, with the largest gain when the image sensor partially fails. If these results hold, event-aided lifting is a low-cost way to harden occupancy prediction for autonomous driving against exactly the conditions that break RGB-only systems.

What carries the argument

The load-bearing mechanism is the Event-aided Lifting Module (ELM), a fusion-based lifting block inserted during 2D-to-3D view transformation. ELM takes separately encoded image and event features, sums their key and value pairs, applies self-attention to the aggregated features to compute a per-location gating weight $w$, blends the two modalities as $(1-w)F_{img}+w F_{event}$, and then uses deformable attention to turn those fused key/value features into 3D voxel queries. The module's job is to do cross-modal fusion at the exact step where 2D features are projected into 3D, preserving spatial fidelity and keeping temporal event cues available to the 3D volume construction.

What would settle it

Run the SemanticKITTI-C shot-noise protocol with real event streams recorded by an event camera synchronized to the RGB image, instead of DVS-Voltmeter-simulated events. If EvSSC's 52.5% relative mIoU gain over VoxFormer-S collapses or becomes negative, then the simulated-event premise, not the fusion mechanism, produced the headline robustness.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the right place to fuse event data with RGB in an SSC network is inside the lifting step, not before or after it. EvSSC builds two encoders, one for images and one for rasterized events; in ELM, the key and value features of both modalities are added, fed through self-attention to produce a scalar gating weight, and then blended into fused key and value features; a deformable-attention stage uses those fused values to query voxel features for the 3D volume. This 'fusion-based lifting' yields higher mIoU than early 2D fusion or late 3D fusion in ablations, and it plugs into both transformer-based (VoxFormer) and LSS-based (SGN) occupancy models. The paper also introduces DSEC-SSC, the first real-world event-aided SSC dataset, with a semi-automatic 4D labeling pipeline that separates static and dynamic objects and reconstructs dynamic instances in time so labels follow moving vehicles and people.

Load-bearing premise

The DSEC-SSC ground-truth labels come from a semi-automatic pipeline that uses manual BEV annotation and linear interpolation for dynamic objects, and most robustness numbers use simulated rather than real events; if either source is unreliable, the reported gains may not transfer to real conditions.

Editorial extensions

If this is right

  • On the simulated SemanticKITTI-E benchmark, EvSSC raises VoxFormer mIoU from 12.86 to 13.61 and SGN from 14.55 to 15.15, with only a millisecond-level latency increase.
  • On corrupted SemanticKITTI-C, event fusion improves mIoU over RGB-only baselines in all five degradations, in both out-of-domain and in-domain settings; VoxFormer-S goes from 8.29 to 12.64 mIoU under in-domain shot noise, a 52.5% relative gain.
  • On real-world DSEC-SSC, adding events raises VoxFormer mIoU from 25.62 to 26.34 and SGN from 29.06 to 29.55, and helps recover vehicles and traffic signs in low-light frames.
  • Because ELM works on both transformer-based and LSS-based occupancy models, event-aided lifting can likely be added to other camera-based 3D occupancy architectures without redesigning the rest of the network.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension the paper does not run is testing the corruption protocol with real event streams rather than DVS-Voltmeter simulation; the headline 52.5% gain comes from simulated events, so a real-event test under matched motion blur and low light would show whether the gain persists under actual sensor noise.
  • The gating weight $w$ in ELM is a soft per-location choice between RGB and event evidence, suggesting the same fusion-based lifting design could transfer to other complementary sensors, such as radar or thermal cameras, whenever their features can be expressed as key/value pairs.
  • The DSEC-SSC labeling pipeline is sensor-agnostic and low-cost, so it could be reused to create event-aided SSC labels from other sparse-LiDAR datasets; a useful validation would be to compare DSEC-SSC dynamic-object labels against a dense or independently annotated reference.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces DSEC-SSC, a real-world event-aided semantic scene completion (SSC) benchmark derived from the DSEC dataset through a semi-automatic 4D labeling pipeline, and proposes EvSSC, a fusion framework whose Event-aided Lifting Module (ELM) injects event features into the 2D-to-3D lifting stage. The method is evaluated on DSEC-SSC and on simulated SemanticKITTI-E/C datasets using VoxFormer and SGN backbones. The central claim is that event-aided lifting yields consistent improvements in mIoU across five degradation modes and both in-domain and out-of-domain settings, with a headline 52.5% relative improvement in the in-domain shot-noise setting.

Significance. If the results hold up, the paper makes a useful contribution: it provides the first real-world event-camera benchmark for SSC, proposes a simple and architecture-agnostic fusion point (during lifting) that is ablated against two alternative fusion paradigms, and demonstrates that event cues can help under image degradation. The commitment to release the dataset and code, the benchmarking of multiple baselines, and the explicit ablation of design choices are strengths. The main significance risk is that the only real-event evidence (Table II) rests on a labeling pipeline whose accuracy is not validated, and the headline robustness results are entirely based on simulated events; both issues are addressable in revision.

major comments (4)
  1. [Section III-B, Table II, Section VI] The DSEC-SSC dynamic-object labels are not validated. Section III-B describes a pipeline that manually annotates 'pipeline' instances in BEV and uses linear interpolation to place dynamic objects when LiDAR misses them, but no inter-annotator agreement, no error analysis, and no independent LiDAR cross-check are reported; the Limitations section (Sec. VI) also concedes that pose-estimation accuracy needs improvement. This is load-bearing because the only real-event support for the central claim is Table II, where the EvSSC gains over RGB-only baselines are +0.72 and +0.49 mIoU. If dynamic-object labels are systematically displaced or missing, these differences could lie within label noise. The authors should validate the labels, for example by comparing a subset against dense manual annotation from the LiDAR sweeps or against an independent sensor, and report per-class label agreement.
  2. [Tables II-IV] No uncertainty estimates or significance tests are reported. Many of the claimed gains are small: on DSEC-SSC the gains are +0.72 and +0.49 mIoU, and in Table IV most in-domain corruption gains are below 0.9 mIoU. Without repeated runs or paired significance testing, the 'consistently improved prediction accuracy' claim is not statistically grounded. Please report mean and standard deviation over at least three seeds for the key comparisons, or provide a significance test on the validation-set metric.
  3. [Section V.A, Section V.C, Table IV] The SemanticKITTI-C robustness protocol does not state whether the event streams are generated from the clean RGB images or from the corrupted RGB images. If the events are generated from clean images, the event branch is never exposed to the degradation, which would make part of the robustness gain a by-construction effect. If the events are generated from corrupted images, the fidelity of DVS-Voltmeter simulation under strong blur, noise, or fog needs at least a discussion, since most quantitative robustness claims in Table IV come from this simulated setting. The authors should specify the exact protocol and, ideally, report results for both clean-generated and corrupted-generated events.
  4. [Abstract, Table IV, Section V-C] The abstract's 52.5% relative improvement is drawn from a single cell of Table IV (in-domain shot noise, VoxFormer-S: 8.29 to 12.64). All other corruption cells show much smaller gains, and the SGN-S cell for the same shot-noise condition is only +0.70 mIoU. While 'up to 52.5%' is literally correct, the claim would be better supported if the text reported the range or median improvement across the ten corruption cells and noted that this large gain is an outlier. Please recontextualize the headline number or provide an explanation for why this particular cell is so much larger than the others.
minor comments (5)
  1. [Section V.B, Table II] The text says 'EvSSC (SGN) boots mIoU by 0.31 compared to its RGB baseline (29.37 vs. 29.06)', but Table II lists the EvSSC (SGN-S) mIoU as 29.55, not 29.37. The word 'boots' should be 'boosts', and the inconsistent SGN/SGN-S naming should be harmonized.
  2. [Section III-C, Eqs. (1)-(3)] Equation (2) is notationally unclear: 'Mg = arg max N X i=1 1(di < eps)' does not define a well-posed optimization over the plane parameters, and the sentence 'Here, 1 is an indicator function... and MS∈MS' is garbled. The plane-fitting objective and the role of the K-means cluster count in Eq. (3) should be stated explicitly, including the values of epsilon and the cluster count used.
  3. [Table V(b), Reference [98]] The HATS entry in Table V(b) cites reference [98], but [98] is 'Recovering accurate 3D human pose in the wild using IMUs and a moving camera', which is not the HATS event representation. The HATS method (Sironi et al., CVPR 2018) should be cited instead.
  4. [Section V.A] The sentence 'we report training results using both 2D images (xrgb) and event data (xevent) as inputs' is ambiguous; the tables actually distinguish RGB-only, event-only, and EvSSC (RGB+event) rows. Please rephrase to describe the three input configurations.
  5. [Section V.A, Table IV] The severity levels used to create SemanticKITTI-C (e.g., the OpenCV or image-processing parameters for motion blur, fog, brightness, darkness, and shot noise) are not specified. These settings should be reported in the paper or in the supplementary material to make the robustness results reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EvSSC's accuracy and robustness gains are measured on held-out benchmarks whose labels were generated independently of the model, and no fitted parameter or self-citation forces the central claim.

full rationale

The paper's central claim is that fusing event features during 2D-to-3D lifting improves semantic scene completion accuracy and robustness. The supporting evidence is empirical evaluation on held-out validation splits of DSEC-SSC and SemanticKITTI-E/C. The DSEC-SSC labels come from a semi-automatic pipeline combining DSEC's 2D semantic maps, LiDAR SLAM, point-cloud clustering, and manual BEV annotation; the SemanticKITTI labels are the external official ground truth. Neither label source depends on EvSSC's predictions, so the reported mIoU gains are not forced by construction. ELM's gating weights and attention parameters are learned from training data, and the paper does not fit any parameter to the target result and then rename it as a prediction. The ablation studies select the fusion paradigm and event representation by validation performance, which is standard model selection rather than circular reasoning. Self-citations appear only in related work and as baseline fusion methods in ablations; none is load-bearing for the central claim. Concerns about DSEC-SSC label reliability or fidelity of simulated events are validity and generalization risks, not instances of circularity. Therefore the derivation chain is self-contained with respect to the claimed conclusions.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on dataset-label reliability, simulated-event fidelity, and standard attention components. No free numerical constants are fitted to the target mIoU; the listed parameters are hand-selected design choices with no sensitivity analysis. No new physical entities are postulated.

free parameters (4)
  • Plane-fitting threshold epsilon (Eq 2) = not specified
    Used to separate ground points from non-ground in label refinement; chosen by hand without sensitivity analysis.
  • K-means cluster count (Eq 3) = not specified
    Number of clusters for semantic label voting in static point cloud purification; chosen by hand.
  • Event representation rasterization parameters = not specified
    Rasterized event counts and timestamps are binned into channels; bin size and polarity handling are not reported but affect fusion quality.
  • Corruption severity levels for SemanticKITTI-C = not specified
    Motion blur, fog, brightness, darkness, and shot noise are applied with unspecified severity, and all robustness scores depend on these choices.
assumptions (5)
  • domain assumption 2D semantic maps released with DSEC are accurate enough to seed point-cloud labeling.
    The label pipeline in Sec III-A projects LiDAR points onto DSEC's 2D semantic map Ilabel to separate static and dynamic points; errors propagate into ground truth.
  • domain assumption Human BEV annotation plus linear interpolation reconstructs dynamic object motion correctly.
    Sec III-B uses manual annotation of 'pipeline' instances and linear interpolation to place missing dynamic objects; no independent validation.
  • domain assumption DVS-Voltmeter simulated events faithfully represent real event camera data for SSC.
    SemanticKITTI-E and SemanticKITTI-C results are produced from simulated events; DSEC-SSC provides real events but many robustness numbers rely on simulation.
  • standard math Attention softmax and deformable attention are standard and correctly implemented.
    Eq 9-13 invoke self-attention and deformable attention without proof; acceptable as standard architecture components.
  • ad hoc to paper The 'pipeline effect' in semantic maps preserves spatiotemporal object information.
    Sec III-B relies on the observation that 3D semantic map artifacts retain temporal info; used to extract dynamic object positions. This is a task-specific heuristic, not established theory.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Event-aided Semantic Scene Completion." pith.science (2026). https://pith.science/paper/XIDGXH3Y

@misc{pith2026250202334,
  author       = {Pith},
  title        = {Pith review of: Event-aided Semantic Scene Completion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XIDGXH3Y}},
  note         = {Machine review of arXiv:2502.02334}
}
read the original abstract

Autonomous driving systems rely on robust 3D scene understanding. Recent advances in Semantic Scene Completion (SSC) for autonomous driving underscore the limitations of RGB-based approaches, which struggle under motion blur, poor lighting, and adverse weather. Event cameras, offering high dynamic range and low latency, address these challenges by providing asynchronous data that complements RGB inputs. We present DSEC-SSC, the first real-world benchmark specifically designed for event-aided SSC, which includes a novel 4D labeling pipeline for generating dense, visibility-aware labels that adapt dynamically to object motion. Our proposed RGB-Event fusion framework, EvSSC, introduces an Event-aided Lifting Module (ELM) that effectively bridges 2D RGB-Event features to 3D space, enhancing view transformation and the robustness of 3D volume construction across SSC models. Extensive experiments on DSEC-SSC and simulated SemanticKITTI-E demonstrate that EvSSC is adaptable to both transformer-based and LSS-based SSC architectures. Notably, evaluations on SemanticKITTI-C demonstrate that EvSSC achieves consistently improved prediction accuracy across five degradation modes and both In-domain and Out-of-domain settings, achieving up to a 52.5% relative improvement in mIoU when the image sensor partially fails. Additionally, we quantitatively and qualitatively validate the superiority of EvSSC under motion blur and extreme weather conditions, where autonomous driving is challenged. The established datasets and our codebase will be made publicly at https://github.com/Pandapan01/EvSSC.

Figures

Figures reproduced from arXiv: 2502.02334 by the authors.

Figure 1
Figure 1. (a) Role of event data in enhancing semantic scene completion: Under challenging lighting conditions, RGB￾based methods struggle to detect low-contrast objects, whereas event data enhances visibility and improves 3D occupancy predictions. (b) Performance comparison on the corrupted SemanticKITTI-C: mIoU results across Out-of-Domain and In-Domain scenarios, with and without event data integration, showing scores unde… view at source ↗
Figure 2
Figure 2. Overview of the occupancy label generation pipeline. The pipeline consists of three main steps: Semantic-Maps￾Guided Dynamic Object Segmentation (Sec. III-A), Dynamic Object 4D Reconstruction (Sec. III-B), and Probability-Guided Voxel Refinement (Sec. III-C). 1E+00 1E+01 1E+02 1E+03 1E+04 1E+05 1E+06 1E+07 1E+08 1 car truck other -vehicle bicycle motorcycle person terrain road sidewalks pole traffic -sign building f… view at source ↗
Figure 3
Figure 3. Overview of label distribution in the DSEC-SSC dataset. The label distribution in the DSEC-SSC dataset is presented with the y-axis plotted on a logarithmic scale. providing reliable spatiotemporal information for dynamic objects MD. As shown in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: (a) Event-image 2D-to-3D fusion paradigms: a spectrum of paradigms for feature fusion: (Left) performs 2D feature fusion after encoding, (Right) performs voxel fusion prior to the segmentation head. (Middle) ELM: fuse 2D or 3D features during the lifting process. (b) E…
Figure 6
Figure 6. Figure 6: Overview of degradation modes in the SemanticKITTI dataset. The degradation modes are illustrated, where clean image (a) serves as the baseline. Fog (b), brightness (c), darkness (d), and motion blur (f) represent degradations influenced by environmental factors or the…
Figure 7
Figure 7. Figure 7: Qualitative results of EvSSC and baseline on DSEC-SSC. EvSSC better captures scene layouts in low-light scenes. TABLE IV: Semantic scene completion results on the corrupted SemanticKITTI-C dataset. Corruption VoxFormer-S EvSSC (VoxFormer) SGN-S EvSSC (SGN) Out-of-domai…
Figure 8
Figure 8. Figure 8: Qualitative results of EvSSC and baseline on SemanticKITTI-C. More visual results of 3D occupancy on SemanticKITTI-C, showing predictions from VoxFormer-S [21] and EvSSC (VoxFormer). The first four rows correspond to the degradation mode of fog, while the last four row…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

99 extracted references · 74 canonical work pages

  1. [1]

    Vision-based 3D occupancy prediction in autonomous driving: A review and outlook,

    Y . Zhang, J. Zhang, Z. Wang, J. Xu, and D. Huang, “Vision-based 3D occupancy prediction in autonomous driving: A review and outlook,” arXiv preprint arXiv:2405.02595 , 2024

  2. [2]

    3D semantic scene completion and occupancy prediction for autonomous driving: A survey,

    G. Xu, W. Liu, Z. Ning, Q. Zhao, S. Cheng, and J. Nie, “3D semantic scene completion and occupancy prediction for autonomous driving: A survey,” in CAIT, 2023

  3. [3]

    A survey on occupancy perception for autonomous driving: The information fusion perspective,

    H. Xu, J. Chen, S. Meng, Y . Wang, and L.-P. Chau, “A survey on occupancy perception for autonomous driving: The information fusion perspective,” Information Fusion, 2025

  4. [4]

    Low-latency automotive vision with event cameras,

    D. Gehrig and D. Scaramuzza, “Low-latency automotive vision with event cameras,” Nature, 2024

  5. [5]

    Robo3D: Towards robust and reliable 3D perception against corruptions,

    L. Kong, Y . Liu, X. Li, R. Chen, W. Zhang, J. Ren, L. Pan, K. Chen, and Z. Liu, “Robo3D: Towards robust and reliable 3D perception against corruptions,” in ICCV, 2023

  6. [6]

    Event-based vision: A survey,

    G. Gallego, T. Delbr ¨uck, G. Orchard, C. Bartolozzi, B. Taba, A. Censi, S. Leutenegger, A. J. Davison, J. Conradt, K. Daniilidis, and D. Scara- muzza, “Event-based vision: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022

  7. [7]

    SemanticKITTI: A dataset for semantic scene understand- ing of LiDAR sequences,

    J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, and J. Gall, “SemanticKITTI: A dataset for semantic scene understand- ing of LiDAR sequences,” in ICCV, 2019

  8. [8]

    SemanticPOSS: A point cloud dataset with large quantity of dynamic instances,

    Y . Pan, B. Gao, J. Mei, S. Geng, C. Li, and H. Zhao, “SemanticPOSS: A point cloud dataset with large quantity of dynamic instances,” in IV, 2020

Show all 99 references
  1. [9]

    Occ3D: A large-scale 3D occupancy prediction benchmark for autonomous driving,

    X. Tian, T. Jiang, L. Yun, Y . Mao, H. Yang, Y . Wang, Y . Wang, and H. Zhao, “Occ3D: A large-scale 3D occupancy prediction benchmark for autonomous driving,” in NeurIPS, 2023. 11

  2. [10]

    OpenOccupancy: A large scale benchmark for surrounding semantic occupancy perception,

    X. Wang, Z. Zhu, W. Xu, Y . Zhang, Y . Wei, X. Chi, Y . Ye, D. Du, J. Lu, and X. Wang, “OpenOccupancy: A large scale benchmark for surrounding semantic occupancy perception,” in ICCV, 2023

  3. [11]

    Scene as occupancy,

    W. Tong, C. Sima, T. Wang, L. Chen, S. Wu, H. Deng, Y . Gu, L. Lu, P. Luo, D. Lin, and H. Li, “Scene as occupancy,” in ICCV, 2023

  4. [12]

    SSCBench: A large-scale 3D semantic scene completion benchmark for autonomous driving,

    Y . Li, S. Li, X. Liu, M. Gong, K. Li, N. Chen, Z. Wang, Z. Li, T. Jiang, F. Yu, Y . Wang, H. Zhao, Z. Yu, and C. Feng, “SSCBench: A large-scale 3D semantic scene completion benchmark for autonomous driving,” arXiv preprint arXiv:2306.09001 , 2023

  5. [13]

    nuCraft: Crafting high resolution 3D semantic occupancy for unified 3D scene understanding,

    B. Zhu, Z. Wang, and H. Li, “nuCraft: Crafting high resolution 3D semantic occupancy for unified 3D scene understanding,” in ECCV, 2024

  6. [14]

    PointSSC: A cooperative vehicle-infrastructure point cloud benchmark for semantic scene completion,

    Y . Yan, B. Liu, J. Ai, Q. Li, R. Wan, and J. Pu, “PointSSC: A cooperative vehicle-infrastructure point cloud benchmark for semantic scene completion,” in ICRA, 2024

  7. [15]

    WildOcc: A benchmark for off-road 3D semantic occupancy prediction,

    H. Zhai, J. Mei, C. Min, L. Chen, F. Zhao, and Y . Hu, “WildOcc: A benchmark for off-road 3D semantic occupancy prediction,” arXiv preprint arXiv:2410.15792, 2024

  8. [16]

    V2VSSC: A 3D semantic scene completion benchmark for perception with vehicle to vehicle communication,

    Y . Zhang, J. Li, K. Luo, Y . Yang, J. Han, N. Liu, D. Qin, P. Han, and C. Xu, “V2VSSC: A 3D semantic scene completion benchmark for perception with vehicle to vehicle communication,” arXiv preprint arXiv:2402.04671, 2024

  9. [17]

    Label- efficient semantic scene completion with scribble annotations,

    S. Wang, J. Yu, W. Li, H. Shi, K. Yang, J. Chen, and J. Zhu, “Label- efficient semantic scene completion with scribble annotations,” in IJCAI, 2024

  10. [18]

    RadarOcc: Robust 3D occupancy prediction with 4D imaging radar,

    F. Ding, X. Wen, Y . Zhu, Y . Li, and C. X. Lu, “RadarOcc: Robust 3D occupancy prediction with 4D imaging radar,” in NeurIPS, 2024

  11. [19]

    K-Radar: 4D radar object detection for autonomous driving in various weather conditions,

    D.-H. Paek, S.-H. Kong, and K. T. Wijaya, “K-Radar: 4D radar object detection for autonomous driving in various weather conditions,” in NeurIPS, 2022

  12. [20]

    OmniHD-Scenes: A next- generation multimodal dataset for autonomous driving,

    L. Zheng, L. Yang, Q. Lin, W. Ai, M. Liu, S. Lu, J. Liu, H. Ren, J. Mo, X. Bai, J. Bai, Z. Ma, and X. Zhu, “OmniHD-Scenes: A next- generation multimodal dataset for autonomous driving,” arXiv preprint arXiv:2412.10734, 2024

  13. [21]

    V oxFormer: Sparse voxel transformer for camera- based 3D semantic scene completion,

    Y . Li, Z. Yu, C. Choy, C. Xiao, J. M. Alvarez, S. Fidler, C. Feng, and A. Anandkumar, “V oxFormer: Sparse voxel transformer for camera- based 3D semantic scene completion,” in CVPR, 2023

  14. [22]

    Camera-based 3D semantic scene completion with sparse guidance network,

    J. Mei, Y . Yang, M. Wang, J. Zhu, J. Ra, Y . Ma, L. Li, and Y . Liu, “Camera-based 3D semantic scene completion with sparse guidance network,” IEEE Transactions on Image Processing , 2024

  15. [23]

    SCPNet: Semantic scene completion on point cloud,

    Z. Xia, Y . Liu, X. Li, X. Zhu, Y . Ma, Y . Li, Y . Hou, and Y . Qiao, “SCPNet: Semantic scene completion on point cloud,” in CVPR, 2023

  16. [24]

    S3CNet: A sparse semantic scene completion network for LiDAR point clouds,

    R. Cheng, C. Agia, Y . Ren, X. Li, and L. Bingbing, “S3CNet: A sparse semantic scene completion network for LiDAR point clouds,” in CoRL, 2021

  17. [25]

    Sparse single sweep LiDAR point cloud segmentation via learning contextual shape priors from scene completion,

    X. Yan, J. Gao, J. Li, R. Zhang, Z. Li, R. Huang, and S. Cui, “Sparse single sweep LiDAR point cloud segmentation via learning contextual shape priors from scene completion,” in AAAI, 2021

  18. [26]

    LMSCNet: Lightweight multiscale 3D semantic completion,

    L. Rold ˜ao, R. de Charette, and A. Verroust-Blondet, “LMSCNet: Lightweight multiscale 3D semantic completion,” in 3DV, 2020

  19. [27]

    TALoS: Enhancing semantic scene completion via test-time adaptation on the line of sight,

    H.-K. Jang, J. Kim, H. Kweon, and K.-J. Yoon, “TALoS: Enhancing semantic scene completion via test-time adaptation on the line of sight,” in NeurIPS, 2024

  20. [28]

    SLCF-Net: Sequential LiDAR-camera fusion for semantic scene completion using a 3D recurrent U-Net,

    H. Cao and S. Behnke, “SLCF-Net: Sequential LiDAR-camera fusion for semantic scene completion using a 3D recurrent U-Net,” in ICRA, 2024

  21. [29]

    MASS: Multi-attentional semantic segmentation of LiDAR data for dense top-view understanding,

    K. Peng, J. Fei, K. Yang, A. Roitberg, J. Zhang, F. Bieder, P. Heidenre- ich, C. Stiller, and R. Stiefelhagen, “MASS: Multi-attentional semantic segmentation of LiDAR data for dense top-view understanding,” IEEE Transactions on Intelligent Transportation Systems , 2022

  22. [30]

    Co-Occ: Coupling explicit feature fusion with volume rendering regularization for multi-modal 3D semantic occupancy prediction,

    J. Pan, Z. Wang, and L. Wang, “Co-Occ: Coupling explicit feature fusion with volume rendering regularization for multi-modal 3D semantic occupancy prediction,” IEEE Robotics and Automation Letters , 2024

  23. [31]

    OccFusion: Multi- sensor fusion framework for 3D semantic occupancy prediction,

    Z. Ming, J. S. Berrio, M. Shan, and S. Worrall, “OccFusion: Multi- sensor fusion framework for 3D semantic occupancy prediction,” IEEE Transactions on Intelligent Vehicles, 2024

  24. [32]

    LiDAR-camera continuous fusion in voxelized grid for semantic scene completion,

    Z. Lu, B. Cao, and Q. Hu, “LiDAR-camera continuous fusion in voxelized grid for semantic scene completion,” IEEE Transactions on Circuits and Systems for Video Technology , 2024

  25. [33]

    Bi-SSC: Geometric-semantic bidirectional fusion for camera-based 3D semantic scene completion,

    Y . Xue, R. Li, F. Wu, Z. Tang, K. Li, and M. Duan, “Bi-SSC: Geometric-semantic bidirectional fusion for camera-based 3D semantic scene completion,” in CVPR, 2024

  26. [34]

    2D semantic-guided semantic scene completion,

    X. Liu, H. Xie, S. Zhang, H. Yao, R. Ji, L. Nie, and D. Tao, “2D semantic-guided semantic scene completion,” International Journal of Computer Vision, 2024

  27. [35]

    RadOcc: Learning cross-modality occupancy knowledge through rendering assisted distillation,

    H. Zhang, X. Yan, D. Bai, J. Gao, P. Wang, B. Liu, S. Cui, and Z. Li, “RadOcc: Learning cross-modality occupancy knowledge through rendering assisted distillation,” in AAAI, 2024

  28. [36]

    LOMA: Language-assisted semantic occupancy network via triplane mamba,

    Y . Cui, Z. Li, J. Wang, and Z. Fang, “LOMA: Language-assisted semantic occupancy network via triplane mamba,” AAAI, 2025

  29. [37]

    MonoScene: Monocular 3D semantic scene completion,

    A.-Q. Cao and R. De Charette, “MonoScene: Monocular 3D semantic scene completion,” in CVPR, 2022

  30. [38]

    OccFormer: Dual-path transformer for vision-based 3D semantic occupancy prediction,

    Y . Zhang, Z. Zhu, and D. Du, “OccFormer: Dual-path transformer for vision-based 3D semantic occupancy prediction,” in ICCV, 2023

  31. [39]

    Symphonize 3D semantic scene completion with contextual instance queries,

    H. Jiang, T. Cheng, N. Gao, H. Zhang, T. Lin, W. Liu, and X. Wang, “Symphonize 3D semantic scene completion with contextual instance queries,” in CVPR, 2024

  32. [40]

    PanoSSC: Exploring monocular panoptic 3D scene reconstruction for autonomous driving,

    Y . Shi, J. Li, K. Jiang, K. Wang, Y . Wang, M. Yang, and D. Yang, “PanoSSC: Exploring monocular panoptic 3D scene reconstruction for autonomous driving,” in 3DV, 2024

  33. [41]

    Not all voxels are equal: Hardness-aware semantic scene completion with self- distillation,

    S. Wang, J. Yu, W. Li, W. Liu, X. Liu, J. Chen, and J. Zhu, “Not all voxels are equal: Hardness-aware semantic scene completion with self- distillation,” in CVPR, 2024

  34. [42]

    FastOcc: Accelerating 3D occupancy prediction by fusing the 2D bird’s-eye view and perspective view,

    J. Hou, X. Li, W. Guan, G. Zhang, D. Feng, Y . Du, X. Xue, and J. Pu, “FastOcc: Accelerating 3D occupancy prediction by fusing the 2D bird’s-eye view and perspective view,” in ICRA, 2024

  35. [43]

    LowRankOcc: Tensor decomposition and low-rank recovery for vision-based 3D semantic occupancy prediction,

    L. Zhao, X. Xu, Z. Wang, Y . Zhang, B. Zhang, W. Zheng, D. Du, J. Zhou, and J. Lu, “LowRankOcc: Tensor decomposition and low-rank recovery for vision-based 3D semantic occupancy prediction,” in CVPR, 2024

  36. [44]

    Re- liOcc: Towards reliable semantic occupancy prediction via uncertainty learning,

    S. Wang, Z. Wang, J. Yu, W. Li, B. Feng, J. Chen, and J. Zhu, “Re- liOcc: Towards reliable semantic occupancy prediction via uncertainty learning,” arXiv preprint arXiv:2409.18026 , 2024

  37. [45]

    AdaptiveOcc: Adaptive octree-based network for multi-camera 3D semantic occupancy prediction in autonomous driving,

    T. Yang, Y . Qian, W. Yan, C. Wang, and M. Yang, “AdaptiveOcc: Adaptive octree-based network for multi-camera 3D semantic occupancy prediction in autonomous driving,” IEEE Transactions on Circuits and Systems for Video Technology, 2024

  38. [46]

    Offboard occupancy refinement with hybrid propagation for autonomous driving,

    H. Shi, S. Wang, J. Zhang, X. Yin, Z. Wang, G. Wang, J. Zhu, K. Yang, and K. Wang, “Offboard occupancy refinement with hybrid propagation for autonomous driving,” arXiv preprint arXiv:2403.08504 , 2024

  39. [47]

    ZOPP: A framework of zero-shot offboard panoptic perception for autonomous driving,

    T. Ma, H. Zhou, Q. Huang, X. Yang, J. Guo, B. Zhang, M. Dou, Y . Qiao, B. Shi, and H. Li, “ZOPP: A framework of zero-shot offboard panoptic perception for autonomous driving,” in NeurIPS, 2024

  40. [48]

    OccWorld: Learning a 3D occupancy world model for autonomous driving,

    W. Zheng, W. Chen, Y . Huang, B. Zhang, Y . Duan, and J. Lu, “OccWorld: Learning a 3D occupancy world model for autonomous driving,” in ECCV, 2024

  41. [49]

    LinkOcc: 3D semantic occupancy prediction with temporal association,

    W. Ouyang, Z. Xu, B. Shen, J. Wang, and Y . Xu, “LinkOcc: 3D semantic occupancy prediction with temporal association,” IEEE Transactions on Circuits and Systems for Video Technology , 2024

  42. [50]

    GaussianFormer: Scene as gaussians for vision-based 3D semantic occupancy prediction,

    Y . Huang, W. Zheng, Y . Zhang, J. Zhou, and J. Lu, “GaussianFormer: Scene as gaussians for vision-based 3D semantic occupancy prediction,” in ECCV, 2024

  43. [51]

    Tri-perspective view for vision-based 3D semantic occupancy prediction,

    ——, “Tri-perspective view for vision-based 3D semantic occupancy prediction,” in CVPR, 2023

  44. [52]

    Occupancy as set of points,

    Y . Shi, T. Cheng, Q. Zhang, W. Liu, and X. Wang, “Occupancy as set of points,” in ECCV, 2024

  45. [53]

    Instance-aware monocular 3D semantic scene completion,

    H. Xiao, H. Xu, W. Kang, and Y . Li, “Instance-aware monocular 3D semantic scene completion,” IEEE Transactions on Intelligent Trans- portation Systems, 2024

  46. [54]

    OPUS: Occupancy prediction using a sparse set,

    J. Wang, Z. Liu, Q. Meng, L. Yan, K. Wang, J. Yang, W. Liu, Q. Hou, and M.-M. Cheng, “OPUS: Occupancy prediction using a sparse set,” in NeurIPS, 2024

  47. [55]

    SparseOcc: Rethinking sparse latent representation for vision-based semantic occupancy prediction,

    P. Tang, Z. Wang, G. Wang, J. Zheng, X. Ren, B. Feng, and C. Ma, “SparseOcc: Rethinking sparse latent representation for vision-based semantic occupancy prediction,” in CVPR, 2024

  48. [56]

    Fully sparse 3D occupancy prediction,

    H. Liu, Y . Chen, H. Wang, Z. Yang, T. Li, J. Zeng, L. Chen, H. Li, and L. Wang, “Fully sparse 3D occupancy prediction,” in NeurIPS, 2024

  49. [57]

    SurroundOcc: Multi-camera 3D occupancy prediction for autonomous driving,

    Y . Wei, L. Zhao, W. Zheng, Z. Zhu, J. Zhou, and J. Lu, “SurroundOcc: Multi-camera 3D occupancy prediction for autonomous driving,” in ICCV, 2023

  50. [58]

    POP-3D: Open-vocabulary 3D occupancy prediction from images,

    A. V obecky, O. Sim ´eoni, D. Hurych, S. Gidaris, A. Bursuc, P. P ´erez, and J. Sivic, “POP-3D: Open-vocabulary 3D occupancy prediction from images,” in NeurIPS, 2024

  51. [59]

    VEON: V ocabulary-enhanced occupancy prediction,

    J. Zheng, P. Tang, Z. Wang, G. Wang, X. Ren, B. Feng, and C. Ma, “VEON: V ocabulary-enhanced occupancy prediction,” in ECCV, 2024

  52. [60]

    SelfOcc: Self- supervised vision-based 3D occupancy prediction,

    Y . Huang, W. Zheng, B. Zhang, J. Zhou, and J. Lu, “SelfOcc: Self- supervised vision-based 3D occupancy prediction,” in CVPR, 2024

  53. [61]

    S4C: Self-supervised semantic scene completion with neural fields,

    A. Hayler, F. Wimbauer, D. Muhle, C. Rupprecht, and D. Cremers, “S4C: Self-supervised semantic scene completion with neural fields,” in 3DV, 2024. 12

  54. [62]

    Context and geometry aware voxel transformer for semantic scene completion,

    Z. Yu, R. Zhang, J. Ying, J. Yu, X. Hu, L. Luo, S. Cao, and H. Shen, “Context and geometry aware voxel transformer for semantic scene completion,” in NeurIPS, 2024

  55. [63]

    HybridOcc: NeRF enhanced transformer-based multi-camera 3D occupancy prediction,

    X. Zhao, B. Chen, M. Sun, D. Yang, Y . Wang, X. Zhang, M. Li, D. Kou, X. Wei, and L. Zhang, “HybridOcc: NeRF enhanced transformer-based multi-camera 3D occupancy prediction,” IEEE Robotics and Automation Letters, 2024

  56. [64]

    COTR: Compact occupancy transformer for vision-based 3D occupancy prediction,

    Q. Ma, X. Tan, Y . Qu, L. Ma, Z. Zhang, and Y . Xie, “COTR: Compact occupancy transformer for vision-based 3D occupancy prediction,” in CVPR, 2024

  57. [65]

    Hierarchical temporal context learning for camera-based semantic scene completion,

    B. Li, J. Deng, W. Zhang, Z. Liang, D. Du, X. Jin, and W. Zeng, “Hierarchical temporal context learning for camera-based semantic scene completion,” in ECCV, 2024

  58. [66]

    ViewFormer: Exploring spatiotemporal modeling for multi-view 3D occupancy per- ception via view-guided transformers,

    J. Li, X. He, C. Zhou, X. Cheng, Y . Wen, and D. Zhang, “ViewFormer: Exploring spatiotemporal modeling for multi-view 3D occupancy per- ception via view-guided transformers,” in ECCV, 2024

  59. [67]

    CMDA: Cross- modality domain adaptation for nighttime semantic segmentation,

    R. Xia, C. Zhao, M. Zheng, Z. Wu, Q. Sun, and Y . Tang, “CMDA: Cross- modality domain adaptation for nighttime semantic segmentation,” in ICCV, 2023

  60. [68]

    CMX: Cross-modal fusion for RGB-X semantic segmentation with transformers,

    J. Zhang, H. Liu, K. Yang, X. Hu, R. Liu, and R. Stiefelhagen, “CMX: Cross-modal fusion for RGB-X semantic segmentation with transformers,” IEEE Transactions on Intelligent Transportation Systems, 2023

  61. [69]

    Event-guided low-light video semantic segmentation,

    Z. Yao and M. C. Chuah, “Event-guided low-light video semantic segmentation,” in WACV, 2025

  62. [70]

    EISNet: A multi-modal fusion network for semantic segmentation with events and images,

    B. Xie, Y . Deng, Z. Shao, and Y . Li, “EISNet: A multi-modal fusion network for semantic segmentation with events and images,” IEEE Transactions on Multimedia , 2024

  63. [71]

    EV-SegNet: Semantic segmentation for event-based cameras,

    I. Alonso and A. C. Murillo, “EV-SegNet: Semantic segmentation for event-based cameras,” in CVPRW, 2019

  64. [72]

    Superevents: Towards native semantic segmentation for event-based cameras,

    W. F. Low, A. Sonthalia, Z. Gao, A. van Schaik, and B. Ramesh, “Superevents: Towards native semantic segmentation for event-based cameras,” in ICONS, 2021

  65. [73]

    ESS: Learn- ing event-based semantic segmentation from still images,

    Z. Sun, N. Messikommer, D. Gehrig, and D. Scaramuzza, “ESS: Learn- ing event-based semantic segmentation from still images,” in ECCV, 2022

  66. [74]

    Event-based semantic segmentation with posterior attention,

    Z. Jia, K. You, W. He, Y . Tian, Y . Feng, Y . Wang, X. Jia, Y . Lou, J. Zhang, G. Li, and Z. Zhang, “Event-based semantic segmentation with posterior attention,” IEEE Transactions on Image Processing , 2023

  67. [75]

    ISSAFE: Improving semantic segmentation in accidents by fusing event-based data,

    J. Zhang, K. Yang, and R. Stiefelhagen, “ISSAFE: Improving semantic segmentation in accidents by fusing event-based data,” in IROS, 2021

  68. [76]

    Exploring event-driven dynamic context for accident scene segmentation,

    ——, “Exploring event-driven dynamic context for accident scene segmentation,” IEEE Transactions on Intelligent Transportation Systems, 2022

  69. [77]

    Accurate and efficient event-based semantic segmentation using adaptive spiking encoder–decoder network,

    R. Zhang, L. Leng, K. Che, H. Zhang, J. Cheng, Q. Guo, J. Liao, and R. Cheng, “Accurate and efficient event-based semantic segmentation using adaptive spiking encoder–decoder network,” IEEE Transactions on Neural Networks and Learning Systems , 2024

  70. [78]

    Spike-BRGNet: Efficient and accurate event-based semantic segmentation with boundary region-guided spiking neural networks,

    X. Long, X. Zhu, F. Guo, C. Chen, X. Zhu, F. Gu, S. Yuan, and C. Zhang, “Spike-BRGNet: Efficient and accurate event-based semantic segmentation with boundary region-guided spiking neural networks,” IEEE Transactions on Circuits and Systems for Video Technology, 2024

  71. [79]

    Embracing events and frames with hierarchical feature refinement network for object detection,

    H. Cao, Z. Zhang, Y . Xia, X. Li, J. Xia, G. Chen, and A. Knoll, “Embracing events and frames with hierarchical feature refinement network for object detection,” in ECCV, 2024

  72. [80]

    Enhancing traffic object detection in variable illumination with RGB-event fusion,

    Z. Liu, N. Yang, Y . Wang, Y . Li, X. Zhao, and F.-Y . Wang, “Enhancing traffic object detection in variable illumination with RGB-event fusion,” IEEE Transactions on Intelligent Transportation Systems , 2024

  73. [81]

    SODFormer: Streaming object detection with transformer using events and frames,

    D. Li, Y . Tian, and J. Li, “SODFormer: Streaming object detection with transformer using events and frames,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023

  74. [82]

    Event-based fusion for motion deblurring with cross- modal attention,

    L. Sun, C. Sakaridis, J. Liang, Q. Jiang, K. Yang, P. Sun, Y . Ye, K. Wang, and L. V . Gool, “Event-based fusion for motion deblurring with cross- modal attention,” in ECCV, 2022

  75. [83]

    Frequency-aware event-based video deblurring for real-world motion blur,

    T. Kim, H. Cho, and K.-J. Yoon, “Frequency-aware event-based video deblurring for real-world motion blur,” in CVPR, 2024

  76. [84]

    Latency correction for event-guided deblurring and frame interpolation,

    Y . Yang, J. Liang, B. Yu, Y . Chen, J. S. Ren, and B. Shi, “Latency correction for event-guided deblurring and frame interpolation,” in CVPR, 2024

  77. [85]

    RPEFlow: Multimodal fusion of RGB-pointcloud-event for joint optical flow and scene flow estimation,

    Z. Wan, Y . Mao, J. Zhang, and Y . Dai, “RPEFlow: Multimodal fusion of RGB-pointcloud-event for joint optical flow and scene flow estimation,” in ICCV, 2023

  78. [86]

    Towards anytime optical flow estimation with event cameras,

    Y . Ye, H. Shi, K. Yang, Z. Wang, X. Yin, Y . Lin, M. Liu, Y . Wang, and K. Wang, “Towards anytime optical flow estimation with event cameras,” arXiv preprint arXiv:2307.05033 , 2023

  79. [87]

    Frame- event alignment and fusion network for high frame rate tracking,

    J. Zhang, Y . Wang, W. Liu, M. Li, J. Bai, B. Yin, and X. Yang, “Frame- event alignment and fusion network for high frame rate tracking,” in CVPR, 2023

  80. [88]

    VisEvent: Reliable object tracking via collaboration of frame and event flows,

    X. Wang, J. Li, L. Zhu, Z. Zhang, Z. Chen, X. Li, Y . Wang, Y . Tian, and F. Wu, “VisEvent: Reliable object tracking via collaboration of frame and event flows,” IEEE Transactions on Cybernetics , 2024

  81. [89]

    Towards robust keypoint detection and tracking: A fusion approach with event-aligned image features,

    X. Wang, H. Yu, L. Yu, W. Yang, and G.-S. Xia, “Towards robust keypoint detection and tracking: A fusion approach with event-aligned image features,” IEEE Robotics and Automation Letters , 2024

  82. [90]

    DSEC: A stereo event camera dataset for driving scenarios,

    M. Gehrig, W. Aarents, D. Gehrig, and D. Scaramuzza, “DSEC: A stereo event camera dataset for driving scenarios,” IEEE Robotics and Automation Letters, 2021

  83. [91]

    FAST-LIO2: Fast direct LiDAR-inertial odometry,

    W. Xu, Y . Cai, D. He, J. Lin, and F. Zhang, “FAST-LIO2: Fast direct LiDAR-inertial odometry,” IEEE Transactions on Robotics , 2022

  84. [92]

    3D semantic scene completion: A survey,

    L. Rold ˜ao, R. de Charette, and A. Verroust-Blondet, “3D semantic scene completion: A survey,” International Journal of Computer Vision, 2022

  85. [93]

    Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3D,

    J. Philion and S. Fidler, “Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3D,” in ECCV, 2020

  86. [94]

    Semantic scene completion from a single depth image,

    S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser, “Semantic scene completion from a single depth image,” in CVPR, 2017

  87. [95]

    BEVDepth: Acquisition of reliable depth for multi-view 3D object detection,

    Y . Li, Z. Ge, G. Yu, J. Yang, Z. Wang, Y . Shi, J. Sun, and Z. Li, “BEVDepth: Acquisition of reliable depth for multi-view 3D object detection,” in AAAI, 2023

  88. [96]

    DVS-V oltmeter: Stochastic process- based event simulator for dynamic vision sensors,

    S. Lin, Y . Ma, Z. Guo, and B. Wen, “DVS-V oltmeter: Stochastic process- based event simulator for dynamic vision sensors,” in ECCV, 2022

  89. [97]

    Frame-free dynamic digital vision,

    T. Delbr ¨uck, “Frame-free dynamic digital vision,” in International Symposium on Secure-Life Electronics, Advanced Electronics for Quality Life and Society , 2008

  90. [98]

    Recovering accurate 3D human pose in the wild using IMUs and a moving camera,

    T. von Marcard, R. Henschel, M. J. Black, B. Rosenhahn, and G. Pons- Moll, “Recovering accurate 3D human pose in the wild using IMUs and a moving camera,” in ECCV, 2018

  91. [99]

    CBAM: Convolutional block attention module,

    S. Woo, J. Park, J.-Y . Lee, and I. S. Kweon, “CBAM: Convolutional block attention module,” in ECCV, 2018

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.