REVIEW 4 major objections 5 minor 114 references
High-Speed Dynamic 3D Imaging with Sensor Fusion Splatting
T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read Combining RGB, event, and depth data reconstructs fast-moving 3D scenes in low light and small baselines.
desk verdict A plausible RGB+event+depth fusion framework for dynamic 3DGS, but the evaluation does not isolate the fusion benefit. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is a set of deformable 3D Gaussians, a point-based scene representation in which each Gaussian carries position, rotation, scale, color, and opacity and is rendered by splatting onto the image plane. A temporal deformation field $\Delta G(\boldsymbol{\mu}, \boldsymbol{s}, \boldsymbol{r}) = M_{\boldsymbol{g},\boldsymbol{\theta}}(\boldsymbol{\mu}, \gamma(t))$ takes a Gaussian's canonical parameters and a time encoding and outputs offsets, so the same Gaussians serve every modality at every timestamp. The mechanism that makes fusion work is the joint optimization: the RGB loss ($L_{\mathrm{rgb}}$) fixes color, the event loss ($L_{\mathrm{event}}$) matches integrated log-intensity change between randomly sampled timestamps with a mask that discards cancelled events, the LPIPS loss ($L_{\mathrm{lpips}}$) aligns renderings to E2VID grayscale reconstructions, the depth loss ($L_{\mathrm{depth}}$) compares rendered z-distances to measured depth, and a second-order smoothness term ($L_{\boldsymbol{g}}$) regularizes the deformation grid. Each loss is a differentiable check on the same Gaussians, so errors from any one sensor pull the shared representation into agreement.
What would settle it
Re-run the real Dancing Toy capture with a deliberately introduced temporal offset between the event camera and the RGB/depth cameras (e.g., 1 ms), keeping the method and all other settings fixed. If PSNR and depth error fall to the same level as the RGB-only baselines for sub-millisecond offsets, the fusion gain is carried by sensor alignment rather than by the shared-Gaussian representation.
Extended reading notes
Core claim
The central claim is that a shared set of deformable 3D Gaussians, optimized simultaneously against RGB images, event streams, and depth maps, is enough to reconstruct fast, non-rigid 3D scenes at arbitrary timestamps. The paper treats each modality as a different projection of the same underlying dynamic scene: RGB gives texture and color, events give dense log-intensity differences at microsecond resolution, and depth pins down structure inside the missing cone of small-baseline stereo. The deformation field maps canonical Gaussians to each time via a grid encoder and MLP, and all five losses—RGB, event log-intensity difference with a cancellation mask, LPIPS against E2VID-reconstructed frames, rendered depth, and grid temporal smoothness—are minimized jointly. The authors report that the fusion outperforms Deformable 3DGS, 4DGS, and Robust e-NeRF on all synthetic scenes and on real captures of a dancing toy and a Newton's cradle, including dark and small-baseline settings.
Load-bearing premise
The load-bearing premise is that the RGB, event, and depth streams are mutually calibrated and temporally aligned closely enough that the per-pixel RGB, depth, event, and perceptual losses all refer to the same 3D point at the same instant; the paper describes the sensors as calibrated but gives no calibration or synchronization procedure, and it acknowledges that misalignment produces wavering and fog-like artifacts.
Editorial extensions
If this is right
- High-speed deforming 3D scenes can be captured with a small multi-sensor rig instead of ultra-high-speed cameras that cost over $100k and record only a few seconds.
- Novel-view and novel-time synthesis becomes possible at arbitrary timestamps because the deformation field is continuous in time.
- Reconstruction remains accurate under low light, high dynamic range, small baseline, and fast motion, conditions where RGB-only or event-only methods visibly fail.
- Fewer RGB training frames are needed: the paper reports strong structural accuracy with as few as 25 or 10 training samples, because events and depth fill in temporal and geometric information.
- The shared representation cuts data volume relative to high-speed video, since only low-frame-rate RGB, asynchronous events, and depth maps are stored.
Reading between the lines
- Extension: the practical bottleneck of this approach is likely calibration and temporal alignment, so an online alignment-estimation variant of the loss is a natural next step.
- Extension: the same shared-Gaussian pattern could absorb other asynchronous depth sensors, such as single-photon SPAD arrays or sonar, by replacing the depth loss with the corresponding measurement model.
- Extension: the use of E2VID frames with LPIPS suggests pretrained event-to-video networks can act as a denoising bridge for event supervision; a testable variant would weight the raw event loss only in regions where E2VID confidence is high.
- Extension: an ablation that removes each sensor loss on identical real sequences would quantify the marginal contribution of each modality; the paper's comparisons are against full-system baselines, so such an ablation is an open test.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes Sensor Fusion Splatting, a deformable 3D Gaussian splatting framework that reconstructs high-speed dynamic scenes from jointly optimized RGB, event, and depth data. The method renders RGB, depth, and log-intensity-difference images from shared Gaussians with a temporal deformation field, trained with a weighted combination of five losses (Eq. 13). The authors evaluate on four synthetic scenes rendered in Blender with ESIM events and on three real-world scenes captured with a custom prototype consisting of a FLIR RGB camera, Prophesee EVK4 event camera, and LUCIID Helios2+ depth camera. Quantitative results in Tables 1 and 2 report consistent improvements in PSNR, LPIPS, and DRMS over Robust e-NeRF, Deformable 3DGS, and 4DGS, with qualitative results in Figures 5-8. The paper claims significant outperformance over state-of-the-art methods and robustness to small baselines, sparse training samples, fast motion, and challenging lighting.
Significance. If the claimed gains were properly isolated, the method would offer a compelling alternative to high-speed cameras for dynamic 3D capture, with applications in robotics, graphics, and computational imaging. The paper's main contributions are a shared Gaussian representation that unifies three sensor modalities, a hardware prototype, and a commitment to release code and data. The authors also honestly report failure modes: Section 3.2 acknowledges that raw events cause wavering artifacts and that E2VID alignment issues induce fog-like artifacts, and Section 4.2 notes color jitter around the eyes. However, the current evidence does not yet demonstrate that sensor fusion specifically, rather than the additional depth and event supervision, drives the reported gains, and the DRMS metric is circular with the depth training signal. The paper's reproducibility promises are not yet verifiable because code and data are not provided.
major comments (4)
- [§4.1, Table 1] The comparison in Table 1 does not isolate the effect of sensor fusion. Deformable 3DGS and 4DGS receive only RGB frames, while the proposed method additionally receives dense depth maps (Eq. 11) and event supervision (Eqs. 8-9); Robust e-NeRF is an event-only baseline. Under this protocol, better PSNR/LPIPS/DRMS is expected even if the fusion machinery contributes nothing, simply because the proposed method is given more information. No ablation in §4.3 removes L_depth or L_event from Eq. (13), and no RGB-only variant of the same deformable-Gaussian architecture is reported. To support the abstract's causal claim about fusion, the authors should add an RGB-only version of the same model and a set of sensor-subset ablations (RGB+depth, RGB+event, all sensors).
- [§4.1, Eq. (11), Tables 1-2] The DRMS metric is partly circular with the training objective. Depth maps 'directly extracted from Blender' (§4.1) are used as supervision in Eq. (11) and also serve as the ground truth for DRMS in Tables 1 and 2. Consequently, the structural accuracy numbers measure, at least partly, how well the model fits the training target rather than generalization of geometry. The authors should report depth error on held-out viewpoints or timestamps not used in training, or evaluate DRMS on a separate depth map.
- [§4.2, Fig. 2] The real-world experiments assume that the RGB, event, and depth streams are mutually calibrated and temporally aligned, but the calibration and synchronization procedure is not described. The paper itself states that calibration errors produce 'wavering artifacts and jumpy background' and that E2VID misalignment creates 'fog-like artifacts' (§3.2), so this assumption is load-bearing for the claimed fusion benefit. Please specify the intrinsic/extrinsic calibration, temporal offset estimation, and synchronization steps, and if possible report calibration or alignment errors or an ablation showing sensitivity to misalignment.
- [§4.1, Tables 1-2] Tables 1 and 2 report single numbers without error bars or statistical tests, and the text does not state whether the objective metrics are computed on training views/timestamps or held-out ones. Since the paper claims novel-view generalization (Figures 3 and 4, and §4.2 interpolation), the evaluation protocol should be explicit. Please report the evaluation split (held-out timestamps/viewpoints) and repeated-run variance for the main metrics.
minor comments (5)
- [§3.2, Eqs. (6)-(8)] The notation Δ𝑳𝑡𝑠 )𝑡𝑒 is nonstandard and difficult to parse; please use a subscript such as Δ𝑳_{t_s→t_e}. In addition, the integral in Eq. (6) over a discrete event stream is not formally defined; writing it as a sum over events in the window would be clearer.
- [§4.3] The text compares with '4DGS [98]' while Table 2 and the earlier experiments cite 4DGS as [91]; please reconcile the reference.
- [§4.2] The phrase 'a a Prophesee EVK4' contains a duplicated article.
- [§4.2] The RGB, event, and depth streams have different resolutions (900×900, 600×600, and 380×380, respectively); the paper does not explain how the per-pixel losses in Eq. (13) are computed across these resolutions. Please specify the resampling or projection.
- [Fig. 3] The DRMS values printed in the Figure 3 caption (0.345, 1.814, 1.343) do not match the Table 1 entries for the same scenes (e.g., Hummingbird Ours is 1.615); please clarify which scene/viewpoint these numbers correspond to or correct the values.
Circularity Check
Depth maps serve both as the training loss (Eq. 11) and as the DRMS evaluation metric, so part of the reported structural-accuracy gain measures the fitted objective rather than an independent test.
-
fitted input called prediction
[Section 3.2 (Eq. 11) and Section 4.1 ('Simulation and Synthetic Data'; 'Experiments and Assessment'), Tables 1 and 2]
"The depth-based loss function is defined as: L𝑑𝑒𝑝𝑡ℎ =|𝑫𝑡− b𝑫𝑡|. ... Depth information was directly extracted from Blender, while events were simulated using ESIM. ... We evaluate our experimental results using several metrics, including per-pixel accuracy (PSNR), perceptual quality (LPIPS)[101], and depth root mean square (DRMS)."
The same depth maps are used in two roles: as the training target in Eq. (11) and as the ground truth for the DRMS metric in Tables 1 and 2. For any viewpoint or timestamp that appears in the training set, minimizing L_depth directly drives down the depth error that DRMS measures. The paper does not state that the tabulated DRMS values are restricted to held-out viewpoints or timestamps; only Figure 3's qualitative depth render is explicitly from an unseen camera viewpoint. Since the RGB-only baselines never receive depth supervision, their larger DRMS values are partly guaranteed by the difference in supervision, not by the fusion machinery. Thus the DRMS evidence for 'structural accuracy' is partially self-defined, though it is not the whole support for the paper's central claim.
full rationale
The core optimization — deformable 3D Gaussians trained jointly against RGB, event, and depth losses — is not circular by construction. The event loss (Eq. 8) compares rendered log-intensity differences with event-derived differences, and the LPIPS loss (Eq. 9) uses E2VID reconstructions as targets; the main PSNR/LPIPS comparisons are against ground-truth RGB frames, so those metrics are not identical to the training objectives. The self-citations for the missing-cone argument ([69], [70]) are background support, not load-bearing derivation steps, and they do not force the reconstruction result. The one concrete circularity is depth: Eq. (11) optimizes rendered depth to match the input depth maps, and the DRMS metric reports the same kind of depth error. Unless all tabulated DRMS values are computed on genuinely held-out views, part of the structural-accuracy advantage is a restatement of which methods received depth supervision. This is a partial circularity confined to one evaluation axis. The novel-view RGB renderings and real-world qualitative results provide independent content, so the paper's fusion claim is not wholly forced. A separate concern, not circularity, is that the synthetic events are generated by ESIM from the same Blender frames used for RGB supervision, so the synthetic fusion inputs are not independent modalities and the comparison to RGB-only baselines is confounded by extra supervision; an RGB-only ablation of the same architecture would be needed to isolate the fusion benefit.
Assumptions & free parameters
free parameters (3)
- loss weights lambda_1 through lambda_5 =
not reported
- event integration window bounds l_min and l_max =
l_min = 1 ms; l_max adjusted per illumination, values not reported
- deformation grid resolution and MLP architecture =
not reported
assumptions (4)
- domain assumption All three sensors are calibrated and temporally synchronized so per-pixel losses refer to the same scene point at the same time.
- domain assumption E2VID-reconstructed frames are reliable supervision signals at arbitrary event timestamps.
- domain assumption A canonical deformable Gaussian model with grid features and an MLP can represent the observed high-speed motion.
- standard math Differentiable Gaussian splatting projection and alpha blending equations from Kerbl et al. are correct and applicable to depth and event rendering.
Cite this review
Pith. "Pith review of High-Speed Dynamic 3D Imaging with Sensor Fusion Splatting." pith.science (2026). https://pith.science/paper/TDYXF4KI
@misc{pith2026250204630,
author = {Pith},
title = {Pith review of: High-Speed Dynamic 3D Imaging with Sensor Fusion Splatting},
year = {2026},
howpublished = {\url{https://pith.science/paper/TDYXF4KI}},
note = {Machine review of arXiv:2502.04630}
}
read the original abstract
Capturing and reconstructing high-speed dynamic 3D scenes has numerous applications in computer graphics, vision, and interdisciplinary fields such as robotics, aerodynamics, and evolutionary biology. However, achieving this using a single imaging modality remains challenging. For instance, traditional RGB cameras suffer from low frame rates, limited exposure times, and narrow baselines. To address this, we propose a novel sensor fusion approach using Gaussian splatting, which combines RGB, depth, and event cameras to capture and reconstruct deforming scenes at high speeds. The key insight of our method lies in leveraging the complementary strengths of these imaging modalities: RGB cameras capture detailed color information, event cameras record rapid scene changes with microsecond resolution, and depth cameras provide 3D scene geometry. To unify the underlying scene representation across these modalities, we represent the scene using deformable 3D Gaussians. To handle rapid scene movements, we jointly optimize the 3D Gaussian parameters and their temporal deformation fields by integrating data from all three sensor modalities. This fusion enables efficient, high-quality imaging of fast and complex scenes, even under challenging conditions such as low light, narrow baselines, or rapid motion. Experiments on synthetic and real datasets captured with our prototype sensor fusion setup demonstrate that our method significantly outperforms state-of-the-art techniques, achieving noticeable improvements in both rendering fidelity and structural accuracy.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Benjamin Attal, Eliot Laidlaw, Aaron Gokaslan, Changil Kim, Christian Richardt, James Tompkin, and Matthew O’Toole. 2021. Törf: Time-of-flight radiance fields for dynamic scene view synthesis. Advances in neural information processing systems 34 (2021), 26289–26301
2021
-
[2]
Mohammadreza Babaee and Shahriar Negahdaripour. 2015. 3-D object modeling from 2-D occluding contour correspondences by opti-acoustic stereo imaging. Computer Vision and Image Understanding 132 (2015), 56–74
2015
-
[3]
Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan. 2021. Mip-nerf: A multiscale represen- tation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF international conference on computer vision . 5855–5864
2021
-
[4]
Alexis Baudron, Zihao W Wang, Oliver Cossairt, and Aggelos K Katsaggelos
-
[5]
Mario Bijelic, Tobias Gruber, Fahim Mannan, Florian Kraus, Werner Ritter, Klaus Dietmayer, and Felix Heide. 2020. Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 11682–11692
2020
-
[6]
Amit Bleiweiss and Michael Werman. 2009. Fusing time-of-flight depth and color for real-time segmentation and tracking. In Workshop on Dynamic 3D Imaging. Springer, 58–69
2009
-
[7]
Vivek Boominathan, Kaushik Mitra, and Ashok Veeraraghavan. 2014. Improving resolution and depth-of-field of light field cameras using a hybrid imaging system. In 2014 IEEE International Conference on Computational Photography (ICCP). IEEE, 1–10. 8 • Zou, Qu, Peng, Boominathan, Pediredla, Chakravarthula
2014
-
[8]
Christian Brandli, Thomas A Mantel, Marco Hutter, Markus A Höpflinger, Raphael Berner, Roland Siegwart, and Tobi Delbruck. 2014. Adaptive pulsed laser line extraction for terrain reconstruction using a dynamic vision sensor. Frontiers in neuroscience 7 (2014), 275
2014
Show all 114 references
-
[9]
Ang Cao and Justin Johnson. 2023. Hexplane: A fast representation for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 130–141
2023
-
[10]
Praneeth Chakravarthula, Jim Aldon D’Souza, Ethan Tseng, Joe Bartusek, and Felix Heide. 2023. Seeing with sound: Long-range acoustic beamforming for multimodal scene understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 982–991
2023
-
[11]
Yuzhou Chen, Zhenyu Li, Dongxu Lyu, Yansong Xu, and Guanghui He. 2025. Neural Rendering Acceleration With Deferred Neural Decoding and Voxel- Centric Data Flow. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (2025)
2025
-
[12]
Zhiqin Chen, Thomas Funkhouser, Peter Hedman, and Andrea Tagliasacchi
-
[13]
Blender Online Community. 2024. Blender - a 3D modelling and rendering package. http://www.blender.org
2024
-
[14]
Mingyue Cui, Yuzhang Zhu, Yechang Liu, Yunchao Liu, Gang Chen, and Kai Huang. 2022. Dense depth-map estimation based on fusion of event camera and sparse LiDAR. IEEE Transactions on Instrumentation and Measurement 71 (2022), 1–11
2022
-
[15]
P Kingma Diederik. 2014. Adam: A method for stochastic optimization. (No Title) (2014)
2014
-
[16]
Yilun Du, Yinan Zhang, Hong-Xing Yu, Joshua B Tenenbaum, and Jiajun Wu
-
[17]
Yuanxing Duan, Fangyin Wei, Qiyu Dai, Yuhang He, Wenzheng Chen, and Baoquan Chen. 2024. 4D Gaussian Splatting: Towards Efficient Novel View Synthesis for Dynamic Scenes. arXiv:2402.03307 [cs.CV]
2024 arXiv
-
[18]
Cheng-De Fan, Chen-Wei Chang, Yi-Ruei Liu, Jie-Ying Lee, Jiun-Long Huang, Yu- Chee Tseng, and Yu-Lun Liu. 2024. SpectroMotion: Dynamic 3D Reconstruction of Specular Scenes. arXiv preprint arXiv:2410.17249 (2024)
2024 arXiv
-
[19]
Ben Fei, Jingyi Xu, Rui Zhang, Qingyuan Zhou, Weidong Yang, and Ying He
-
[20]
Di Feng, Christian Haase-Schütz, Lars Rosenbaum, Heinz Hertlein, Claudius Glaeser, Fabian Timm, Werner Wiesbeck, and Klaus Dietmayer. 2020. Deep multi- modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges.IEEE Transactions ...
2020
-
[21]
Fausto Ferreira, Diogo Machado, Gabriele Ferri, Samantha Dugelay, and John Potter. 2016. Underwater optical and acoustic imaging: A time for fusion? a brief overview of the state-of-the-art. OCEANS 2016 MTS/IEEE Monterey (2016), 1–6. https://doi.org/10.1109/OCEANS.2016.7761354
2016
-
[22]
Justin Folden, Atul Ingle, and Sanjeev J Koppal. 2024. FoveaSPAD: Exploit- ing Depth Priors for Adaptive and Efficient Single-Photon 3D Imaging. IEEE Transactions on Computational Imaging 10 (2024), 1722–1735
2024
-
[23]
Sarah Friday, Yunzi Shi, Yaswanth Cherivirala, Vishwanath Saragadam, and Adithya Pediredla. 2024. Snapshot Lidar: Fourier embedding of amplitude and phase for single-image depth reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...
2024
-
[24]
Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. 2023. K-planes: Explicit radiance fields in space, time, and appearance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 12479–12488
2023
-
[25]
Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. 2022. Plenoxels: Radiance fields without neural net- works. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5501–5510
2022
-
[26]
Guillermo Gallego, Tobi Delbrück, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J Davison, Jörg Conradt, Kostas Daniilidis, et al. 2020. Event-based vision: A survey. IEEE transactions on pattern analysis and machine intelligence 44, ...
2020
-
[27]
Chen Gao, Ayush Saraf, Johannes Kopf, and Jia-Bin Huang. 2021. Dynamic view synthesis from dynamic monocular video. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 5712–5721
2021
-
[28]
Quankai Gao, Qiangeng Xu, Zhe Cao, Ben Mildenhall, Wenchao Ma, Le Chen, Danhang Tang, and Ulrich Neumann. 2024. Gaussianflow: Splatting gaussian dynamics for 4d content creation. arXiv preprint arXiv:2403.12365 (2024)
2024 arXiv
-
[29]
Daniel Gehrig, Michelle Rüegg, Mathias Gehrig, Javier Hidalgo-Carrió, and Davide Scaramuzza. 2021. Combining events and frames using recurrent asyn- chronous multimodal networks for monocular depth prediction. IEEE Robotics and Automation Letters 6, 2 (2021), 2822–2829
2021
-
[30]
Richard Grover, Graham Brooker, and Hugh F Durrant-Whyte. 2001. A low level fusion of millimeter wave radar and night-vision imaging for enhanced characterization of a cluttered environment. In Proceedings 2001 Australian Conference on Robotics and Automation
2001
-
[31]
Yijia Guo, Liwen Hu, Lei Ma, and Tiejun Huang. 2024. SpikeGS: Reconstruct 3D scene via fast-moving bio-inspired sensors. arXiv preprint arXiv:2407.03771 (2024)
2024 arXiv
-
[32]
Anant Gupta, Atul Ingle, and Mohit Gupta. 2019. Asynchronous single-photon 3D imaging. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 7909–7918
2019
-
[33]
Anant Gupta, Atul Ingle, Andreas Velten, and Mohit Gupta. 2019. Photon- flooded single-photon 3d cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 6770–6779
2019
-
[34]
Jin Han, Chu Zhou, Peiqi Duan, Yehui Tang, Chang Xu, Chao Xu, Tiejun Huang, and Boxin Shi. 2020. Neuromorphic camera guided high dynamic range imag- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1730–1739
2020
-
[35]
Chen Haoyu, Teng Minggui, Shi Boxin, Wang YIzhou, and Huang Tiejun. 2020. Learning to deblur and generate high frame rate video with an event camera. arXiv preprint arXiv:2003.00847 (2020)
2020 arXiv
-
[36]
Yu He and Shengyong Chen. 2019. Recent advances in 3D data acquisition and processing by time-of-flight camera. IEEE Access 7 (2019), 12495–12510
2019
-
[37]
Peter Henry, Michael Krainin, Evan Herbst, Xiaofeng Ren, and Dieter Fox. 2012. RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments. The international journal of Robotics Research 31, 5 (2012), 647–663
2012
-
[38]
Javier Hidalgo-Carrió, Daniel Gehrig, and Davide Scaramuzza. 2020. Learning monocular dense depth from events. In 2020 International Conference on 3D Vision (3DV). IEEE, 534–542
2020
-
[39]
Radu Horaud, Miles Hansard, Georgios Evangelidis, and Clément Ménier. 2016. An overview of depth cameras and range scanners based on time-of-flight technologies. Machine vision and applications 27, 7 (2016), 1005–1020
2016
-
[40]
Inwoo Hwang, Junho Kim, and Young Min Kim. 2023. Ev-nerf: Event based neural radiance field. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 837–847
2023
-
[41]
Sacha Jungerman, Atul Ingle, Yin Li, and Mohit Gupta. 2022. 3D scene inference from transient histograms. In European Conference on Computer Vision . Springer, 401–417
2022
-
[42]
Moritz Kappel, Florian Hahlbohm, Timon Scholz, Susana Castillo, Christian Theobalt, Martin Eisemann, Vladislav Golyanik, and Marcus Magnor. 2024. D-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from Monocular Video. arXiv preprint arXiv:2406.10078 (2024)
2024 arXiv
-
[43]
Nikhil Keetha, Jay Karhade, Krishna Murthy Jatavallabhula, Gengshan Yang, Sebastian Scherer, Deva Ramanan, and Jonathon Luiten. 2024. SplaTAM: Splat Track & Map 3D Gaussians for Dense RGB-D SLAM. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognit...
2024
-
[44]
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis
-
[45]
Bernhard Kerbl, Andreas Meuleman, Georgios Kopanas, Michael Wimmer, Alexandre Lanvin, and George Drettakis. 2024. A Hierarchical 3D Gaussian Rep- resentation for Real-Time Rendering of Very Large Datasets. ACM Transactions on Graphics 43, 4 (2024)
2024
-
[46]
Hanme Kim, Ankur Handa, Ryad Benosman, Sio-Hoi Ieng, and Andrew J Davison
-
[47]
Hanme Kim, Stefan Leutenegger, and Andrew J Davison. 2016. Real-time 3D reconstruction and 6-DoF tracking with an event camera. In Computer Vision– ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VI 14 . Springer, 349–364
2016
-
[48]
Young Min Kim, Christian Theobalt, James Diebel, Jana Kosecka, Branislav Miscusik, and Sebastian Thrun. 2009. Multi-view image and tof sensor fusion for dense 3d reconstruction. In 2009 IEEE 12th international conference on computer vision workshops, ICCV workshops . IEEE, 1542–1549
2009
-
[49]
ACM Trans
3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Trans. Graph. 42, 4 (2023), 139–1
2023
-
[50]
Jonas Kulhanek and Torsten Sattler. 2023. Tetra-nerf: Representing neural radiance fields using tetrahedra. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 18458–18469
2023
-
[51]
Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. 2021. Neural scene flow fields for space-time view synthesis of dynamic scenes. InProceedings of the High-Speed Dynamic 3D Imaging with Sensor Fusion Splatting • 9 IEEE/CVF Conference on Computer Vision and Pattern Reco...
2021
-
[52]
Ming Liang, Bin Yang, Yun Chen, Rui Hu, and Raquel Urtasun. 2019. Multi- task multi-sensor fusion for 3d object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 7345–7353
2019
-
[53]
David B Lindell, Matthew O’Toole, and Gordon Wetzstein. 2018. Single-photon 3D imaging with deep sensor fusion. ACM Trans. Graph. 37, 4 (2018), 113
2018
-
[54]
David B Lindell, Gordon Wetzstein, and Vladlen Koltun. 2019. Acoustic non- line-of-sight imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 6780–6789
2019
-
[55]
Simon Klenk, Lukas Koestler, Davide Scaramuzza, and Daniel Cremers. 2023. E-nerf: Neural radiance fields from a moving event camera. IEEE Robotics and Automation Letters 8, 3 (2023), 1587–1594
2023
-
[56]
Stephen Lombardi, Tomas Simon, Jason Saragih, Gabriel Schwartz, Andreas Lehrmann, and Yaser Sheikh. 2019. Neural volumes: Learning dynamic render- able volumes from images. arXiv preprint arXiv:1906.07751 (2019)
2019 arXiv
-
[57]
Weng Fei Low and Gim Hee Lee. 2023. Robust e-nerf: Nerf from sparse & noisy events under non-uniform motion. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 18335–18346
2023
-
[58]
Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. 2024. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In 2024 International Conference on 3D Vision (3DV) . IEEE, 800–809
2024
-
[59]
Qi Ma, Danda Pani Paudel, Ajad Chhatkuli, and Luc Van Gool. 2023. Deformable neural radiance fields using rgb and event cameras. InProceedings of the IEEE/CVF International Conference on Computer Vision . 3590–3600
2023
-
[60]
Nathan Matsuda, Oliver Cossairt, and Mohit Gupta. 2015. Mc3d: Motion contrast 3d scanning. In2015 IEEE International Conference on Computational Photography (ICCP). IEEE, 1–10
2015
-
[61]
Jia-Wei Liu, Yan-Pei Cao, Weijia Mao, Wenqiao Zhang, David Junhao Zhang, Jussi Keppo, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. 2022. Devrf: Fast deformable voxel radiance fields for dynamic scenes. Advances in Neural Infor- mation Processing Systems 35 (2022), 36762–36775
2022
-
[62]
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. 2022. In- stant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG) 41, 4 (2022), 1–15
2022
-
[63]
Ramin Nabati and Hairong Qi. 2021. Centerfusion: Center-based radar and camera fusion for 3d object detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision . 1527–1536
2021
-
[64]
Mark Nishimura, David B Lindell, Christopher Metzler, and Gordon Wetzstein
-
[65]
Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz, and Ricardo Martin-Brualla. 2021. Nerfies: Deformable neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 5865–5874
2021
-
[66]
Ryan Po, Adithya Pediredla, and Ioannis Gkioulekas. 2022. Adaptive gating for single-photon 3d imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 16354–16363
2022
-
[67]
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. 2021. Nerf: Representing scenes as neural radiance fields for view synthesis. Commun. ACM 65, 1 (2021), 99–106
2021
-
[68]
Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer
-
[69]
Mohamad Qadri, Kevin Zhang, Akshay Hinduja, Michael Kaess, Adithya Pediredla, and Christopher A Metzler. 2024. AONeuS: A Neural Rendering Framework for Acoustic-Optical Sensor Fusion. arXiv preprint arXiv:2402.03309 (2024)
2024 arXiv
-
[70]
Ziyuan Qu, Omkar Vengurlekar, Mohamad Qadri, Kevin Zhang, Michael Kaess, Christopher Metzler, Suren Jayasuriya, and Adithya Pediredla. 2024. Z-Splat: Z-Axis Gaussian Splatting for Camera-Sonar Fusion. arXiv preprint arXiv:2404.04687 (2024)
2024 arXiv
-
[71]
In European Conference on Computer Vision
Disambiguating monocular depth estimation with a single transient. In European Conference on Computer Vision . Springer, 139–155
-
[72]
Henri Rebecq, Guillermo Gallego, Elias Mueggler, and Davide Scaramuzza. 2018. EMVS: Event-based multi-view stereo—3D reconstruction with an event camera in real-time. International Journal of Computer Vision 126, 12 (2018), 1394–1414
2018
-
[73]
Henri Rebecq, Daniel Gehrig, and Davide Scaramuzza. 2018. ESIM: an open event camera simulator. In Conference on robot learning . PMLR, 969–982
2018
-
[75]
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
D-NeRF: Neural Radiance Fields for Dynamic Scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
-
[76]
Viktor Rudnev, Mohamed Elgharib, Christian Theobalt, and Vladislav Golyanik
-
[77]
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
D-nerf: Neural radiance fields for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 10318–10327
-
[78]
Ruizhi Shao, Zerong Zheng, Hanzhang Tu, Boning Liu, Hongwen Zhang, and Yebin Liu. 2023. Tensor4d: Efficient neural 4d decomposition for high-fidelity dynamic reconstruction and rendering. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 16632–16642
2023
-
[79]
Shikhar Shrestha, Felix Heide, Wolfgang Heidrich, and Gordon Wetzstein. 2016. Computational imaging with multi-camera time-of-flight systems. ACM Trans- actions on Graphics (ToG) 35, 4 (2016), 1–11
2016
-
[80]
Yaadhav Raaj, Alex John, and Tan Jin. 2016. 3D Object Localization using Forward Looking Sonar (FLS) and Optical Camera via particle filter based calibration and fusion. OCEANS 2016 MTS/IEEE Monterey (2016), 1–10. https://doi.org/10.1109/ OCEANS.2016.7761077
2016
-
[81]
Lei Sun, Christos Sakaridis, Jingyun Liang, Qi Jiang, Kailun Yang, Peng Sun, Yaozu Ye, Kaiwei Wang, and Luc Van Gool. 2022. Event-based fusion for motion deblurring with cross-modal attention. InEuropean conference on computer vision. Springer, 412–428
2022
-
[82]
Richard Szeliski. 2022. Computer vision: algorithms and applications . Springer Nature
2022
-
[83]
Henri Rebecq, Timo Horstschäfer, Guillermo Gallego, and Davide Scaramuzza
-
[84]
Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Christoph Lassner, and Christian Theobalt. 2021. Non-rigid neural radiance fields: Recon- struction and novel view synthesis of a dynamic scene from monocular video. In Proceedings of the IEEE/CVF Internation...
2021
-
[85]
Henri Rebecq, René Ranftl, Vladlen Koltun, and Davide Scaramuzza. 2019. High speed and high dynamic range video with an event camera. IEEE transactions on pattern analysis and machine intelligence 43, 6 (2019), 1964–1980
2019
-
[86]
Stepan Tulyakov, Daniel Gehrig, Stamatios Georgoulis, Julius Erbach, Mathias Gehrig, Yuanyou Li, and Davide Scaramuzza. 2021. Time lens: Event-based video frame interpolation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 16155–16164
2021
-
[87]
InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Eventnerf: Neural radiance fields from a single colour event camera. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 4992–5002
-
[88]
Cedric Scheerlinck, Nick Barnes, and Robert Mahony. 2018. Continuous-time intensity estimation using event cameras. In Asian Conference on Computer Vision. Springer, 308–324
2018
-
[89]
Ziyun Wang, Kenneth Chaney, and Kostas Daniilidis. 2022. Evac3d: From event- based apparent contours to 3d models via continuous visual hulls. In European conference on computer vision . Springer, 284–299
2022
-
[90]
Stefan Williams and Ian Mahon. 2004. Simultaneous localisation and mapping on the great barrier reef. IEEE International Conference on Robotics and Automation,
2004
-
[91]
Liangchen Song, Anpei Chen, Zhong Li, Zhang Chen, Lele Chen, Junsong Yuan, Yi Xu, and Andreas Geiger. 2023. Nerfplayer: A streamable dynamic scene representation with decomposed neural radiance fields. IEEE Transactions on Visualization and Computer Graphics 29, 5 (2023), 2732–2742
2023
-
[92]
Wenqi Xian, Jia-Bin Huang, Johannes Kopf, and Changil Kim. 2021. Space-time neural irradiance fields for free-viewpoint video. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 9421–9431
2021
-
[93]
Liuyue Xie, Joel Julin, Koichiro Niinuma, and Laszlo A Jeni. 2024. Gaussian Splatting LK. arXiv preprint arXiv:2407.11309 (2024)
2024 arXiv
-
[94]
Ayush Tewari, Justus Thies, Ben Mildenhall, Pratul Srinivasan, Edgar Tretschk, Wang Yifan, Christoph Lassner, Vincent Sitzmann, Ricardo Martin-Brualla, Stephen Lombardi, et al . 2022. Advances in neural rendering. In Computer Graphics Forum, Vol. 41. Wiley Online Library, 703–735
2022
-
[95]
Tianyi Xiong, Jiayi Wu, Botao He, Cornelia Fermuller, Yiannis Aloimonos, Heng Huang, and Christopher A Metzler. 2024. Event3DGS: Event-based 3D Gaussian Splatting for Fast Egomotion. arXiv preprint arXiv:2406.02972 (2024)
2024 arXiv
-
[96]
Stepan Tulyakov, Alfredo Bochicchio, Daniel Gehrig, Stamatios Georgoulis, Yuanyou Li, and Davide Scaramuzza. 2022. Time lens++: Event-based frame interpolation with parametric non-linear flow and multi-scale fusion. In Pro- ceedings of the IEEE/CVF Conference on Computer Visio...
2022
-
[97]
Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. 2024. Deformable 3d gaussians for high-fidelity monocular dynamic scene 10 • Zou, Qu, Peng, Boominathan, Pediredla, Chakravarthula reconstruction. In Proceedings of the IEEE/CVF Conference on Compute...
2024
-
[98]
SM Nadim Uddin, Soikat Hasan Ahmed, and Yong Ju Jung. 2022. Unsupervised deep event stereo for depth estimation.IEEE Transactions on Circuits and Systems for Video Technology 32, 11 (2022), 7489–7504
2022
-
[99]
Feng Wang, Zilong Chen, Guokang Wang, Yafei Song, and Huaping Liu. 2024. Masked space-time hash encoding for efficient dynamic scene reconstruction. Advances in Neural Information Processing Systems 36 (2024)
2024
-
[100]
Kai Zhang, Gernot Riegler, Noah Snavely, and Vladlen Koltun. 2020. Nerf++: Analyzing and improving neural radiance fields. arXiv preprint arXiv:2010.07492 (2020)
2020 arXiv
-
[101]
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang
-
[103]
Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 2024. 4d gaussian splatting for real- time dynamic scene rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 20310–20320
2024
-
[106]
Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, and Srinath Sridhar
-
[109]
Ziyi Yang, Xinyu Gao, Yangtian Sun, Yihua Huang, Xiaoyang Lyu, Wen Zhou, Shaohui Jiao, Xiaojuan Qi, and Xiaogang Jin. 2024. Spec-gaussian: Anisotropic view-dependent appearance for 3d gaussian splatting.arXiv preprint arXiv:2402.15870 (2024)
2024 arXiv
-
[111]
Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. 2024. Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting. International Conference on Learning Representations (ICLR)
2024
-
[112]
Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. 2024. Mip-Splatting: Alias-free 3D Gaussian Splatting. Conference on Computer Vision and Pattern Recognition (CVPR) (2024)
2024
-
[2004]
Proceedings. ICRA’04. 2004 2 (2004), 1771–1776
2004
-
[2008]
Simultaneous mosaicing and tracking with an event camera. J. Solid State Circ 43 (2008), 566–576
2008
-
[2016]
IEEE Robotics and Automation Letters 2, 2 (2016), 593–600
Evo: A geometric approach to event-based 6-dof parallel tracking and mapping in real time. IEEE Robotics and Automation Letters 2, 2 (2016), 593–600
2016
-
[2018]
In Proceedings of the IEEE conference on computer vision and pattern recognition
The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition . 586–595. High-Speed Dynamic 3D Imaging with Sensor Fusion Splatting • 11 Ground TruthOurs4DGSDeformable 3DGS Fig. 5. Synth...
-
[2020]
arXiv preprint arXiv:2012.05214 (2020)
E3d: event-based 3d shape reconstruction. arXiv preprint arXiv:2012.05214 (2020)
2020 arXiv
-
[2021]
In 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Neural radiance flow for 4d view synthesis and video processing. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . IEEE Computer Society, 14304–14314
2021
-
[2022]
InComputer Graphics Forum, Vol
Neural fields in visual computing and beyond. InComputer Graphics Forum, Vol. 41. Wiley Online Library, 641–676
-
[2023]
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 16569–16578
-
[2024]
arXiv preprint arXiv:2402.07181 (2024)
3d gaussian as a new vision era: A survey. arXiv preprint arXiv:2402.07181 (2024)
2024 arXiv
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.