Horizon3D: Sparse Radar-Camera Fusion for Long-Range 3D Perception in Autonomous Driving
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-01 06:14 UTCgrok-4.3pith:MV5U4Y2Nrecord.jsonopen to challenge →
The pith
Horizon3D fuses radar and camera keypoints into Gaussian primitives that are refined and splatted onto sparse BEV features to lift long-range 3D detection.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Horizon3D initializes Gaussian primitives at radar- and camera-estimated object keypoints using Keypoint-Guided Gaussian Initialization, refines them through Object-Centric Sparse Fusion, and splats them onto the BEV plane to fuse object-level detail with sparse radar BEV context; it further aggregates temporal cues via Dual-Path Temporal Fusion along a BEV path for scene accumulation and a Gaussian path for object motion propagation.
What carries the argument
Keypoint-Guided Gaussian Initialization combined with Object-Centric Sparse Fusion and Dual-Path Temporal Fusion
Load-bearing premise
Radar and camera keypoint estimates are accurate enough and complementary on the TruckScenes validation set to preserve fine object detail while adding scene context.
What would settle it
A direct measurement on TruckScenes showing that the radar and camera keypoint estimates have high error or low complementarity, such that removing or degrading them eliminates the reported NDS and mAP gains.
Figures
read the original abstract
Long-range 3D object detection is critical for safe autonomous driving at highway speeds, yet existing radar-camera fusion methods remain limited at extended ranges. BEV-based methods capture scene-level context but incur rapidly growing computation and often lose fine-grained object detail, while query-based methods are efficient but provide limited scene-level context. Temporal fusion further requires both multi-frame accumulation for sparse distant observations and object-level motion modeling for fast-moving objects. We propose Horizon3D, a sparse radar-camera fusion framework for long-range 3D object detection that combines Gaussian primitives with sparse BEV features. Horizon3D initializes Gaussian primitives at radar- and camera-estimated object keypoints using Keypoint-Guided Gaussian Initialization, refines them through Object-Centric Sparse Fusion, and splats them onto the BEV plane to fuse object-level detail with sparse radar BEV context. It further introduces Dual-Path Temporal Fusion, which aggregates temporal cues through a BEV path for scene-level accumulation and a Gaussian path for object-level motion propagation. Experiments on TruckScenes show that Horizon3D achieves state-of-the-art radar-camera 3D detection performance. On the validation set, it outperforms the previous best method by +3.0 NDS and +1.6 mAP while maintaining competitive inference speed.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. Horizon3D proposes a sparse radar-camera fusion framework for long-range 3D object detection. It initializes Gaussian primitives at radar- and camera-estimated object keypoints via Keypoint-Guided Gaussian Initialization, refines them through Object-Centric Sparse Fusion, splats them onto the BEV plane to combine object detail with sparse radar context, and employs Dual-Path Temporal Fusion (BEV path for scene accumulation and Gaussian path for object motion). Experiments on TruckScenes claim state-of-the-art radar-camera 3D detection, outperforming the prior best by +3.0 NDS and +1.6 mAP on the validation set at competitive speed.
Significance. If substantiated, the approach could meaningfully advance long-range perception by addressing the trade-off between fine object detail and scene-level context in radar-camera fusion while handling temporal sparsity for distant objects.
major comments (2)
- Abstract: The central performance claim (+3.0 NDS and +1.6 mAP gains) is stated without any reference to experimental protocol, validation split details, baseline re-implementations, error bars, or statistical significance, making the SOTA assertion unverifiable from the manuscript text.
- Method section (Keypoint-Guided Gaussian Initialization and Object-Centric Sparse Fusion): The premise that these components successfully preserve object detail while incorporating long-range context rests on the untested assumption that radar- and camera-derived keypoints are sufficiently accurate (low localization error relative to object scale) and complementary on TruckScenes. No quantitative keypoint error metrics (e.g., mean distance to GT centers) or ablations isolating their contribution are supplied.
Simulated Author's Rebuttal
We thank the referee for their detailed and constructive feedback. We address each major comment point-by-point below, indicating where revisions will be made to strengthen the manuscript.
read point-by-point responses
-
Referee: Abstract: The central performance claim (+3.0 NDS and +1.6 mAP gains) is stated without any reference to experimental protocol, validation split details, baseline re-implementations, error bars, or statistical significance, making the SOTA assertion unverifiable from the manuscript text.
Authors: We agree that the abstract would benefit from greater specificity to allow readers to assess the claims directly. In the revised manuscript we will expand the abstract to explicitly state that results are reported on the TruckScenes validation set, that all baselines were re-implemented using their official code and recommended protocols, and that the reported gains are measured under the standard evaluation protocol described in Section 4. We will also note that error bars are omitted following common practice in the field when single-run results are presented, while directing readers to the full experimental details. revision: yes
-
Referee: Method section (Keypoint-Guided Gaussian Initialization and Object-Centric Sparse Fusion): The premise that these components successfully preserve object detail while incorporating long-range context rests on the untested assumption that radar- and camera-derived keypoints are sufficiently accurate (low localization error relative to object scale) and complementary on TruckScenes. No quantitative keypoint error metrics (e.g., mean distance to GT centers) or ablations isolating their contribution are supplied.
Authors: The referee is correct that the current manuscript does not supply direct quantitative keypoint localization errors or an ablation that isolates the Keypoint-Guided Gaussian Initialization step. While the overall performance improvements and the existing fusion ablations provide supporting evidence, we will add both a quantitative keypoint accuracy analysis (reporting mean distance to ground-truth centers for radar- and camera-derived keypoints) and a dedicated ablation isolating the initialization component in the revised version. revision: yes
Circularity Check
No circularity detected; derivation is self-contained
full rationale
The manuscript presents Horizon3D as a proposed architecture (Keypoint-Guided Gaussian Initialization, Object-Centric Sparse Fusion, Dual-Path Temporal Fusion) whose performance is asserted via empirical results on the TruckScenes validation set. No equations, fitted parameters renamed as predictions, self-citations used as load-bearing uniqueness theorems, or ansatzes smuggled via prior work appear in the provided text. The central claims therefore do not reduce to their own inputs by construction and remain externally falsifiable.
Axiom & Free-Parameter Ledger
invented entities (1)
-
Gaussian primitives
no independent evidence
Reference graph
Works this paper leans on
-
[1]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Alibeigi, M., Ljungbergh, W., Tonderski, A., Hess, G., Lilja, A., Lindström, C., Motorniuk, D., Fu, J., Widahl, J., Petersson, C.: Zenseact Open Dataset: A large- scale and diverse multimodal dataset for autonomous driving. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 20178–20188 (2023)
work page 2023
-
[2]
IEEE Robotics and Automation Letters10(1), 828–835 (2025)
Bai, X., Yu, Z., Zheng, L., Zhang, X., Zhou, Z., Zhang, X., Wang, F., Bai, J., Shen, H.L.: SGDet3D: Semantics and geometry fusion for 3d object detection using 4d radar and camera. IEEE Robotics and Automation Letters10(1), 828–835 (2025)
work page 2025
-
[3]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2026) 16 G
Bai, X., Zhou, C., Zheng, L., Liu, J., Cao, S.Y., Zhang, X., Li, Y., Zhang, Z., Shen, H.L.: RaGS: Unleashing 3D gaussian splatting from 4D radar and monocular cue for 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2026) 16 G. Bang et al
work page 2026
-
[4]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Bang, G., Seong, M., Kim, J., Baek, G., Oh, D., Kim, J., Koh, J., Choi, J.W.: RCT- Distill: Cross-modal knowledge distillation framework for radar-camera 3D object detection with temporal fusion. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 25315–25324 (2025)
work page 2025
-
[5]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., Beijbom, O.: nuScenes: A multimodal dataset for autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11621–11631 (2020)
work page 2020
-
[6]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Chen, Y., Liu, J., Zhang, X., Qi, X., Jia, J.: VoxelNeXt: Fully sparse voxelnet for 3d object detection and tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 21674–21683 (2023)
work page 2023
-
[7]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Chu, X., Deng, J., You, G., Duan, Y., Li, H., Zhang, Y.: RaCFormer: Towards high- quality 3d object detection via query-based radar-camera fusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 17081–17091 (2025)
work page 2025
-
[8]
Advances in Neural Information Processing Systems35, 351–363 (2022)
Fan, L., Wang, F., Wang, N., Zhang, Z.X.: Fully sparse 3d object detection. Advances in Neural Information Processing Systems35, 351–363 (2022)
work page 2022
-
[9]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops
Fent, F., Bauerschmidt, P., Lienkamp, M.: RadarGNN: Transformation invariant graph neural network for radar-based perception. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 182–191 (2023)
work page 2023
-
[10]
Advances in Neural Information Processing Systems37, 62062–62082 (2024)
Fent, F., Kuttenreich, F., Ruch, F., Rizwin, F., Juergens, S., Lechermann, L., Nissler, C., Perl, A., Voll, U., Yan, M., Lienkamp, M.: MAN TruckScenes: a multimodal dataset for autonomous trucking in diverse conditions. Advances in Neural Information Processing Systems37, 62062–62082 (2024)
work page 2024
-
[11]
In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 770–778 (2016)
work page 2016
-
[12]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Huang, Y., Thammatadatrakoon, A., Zheng, W., Zhang, Y., Du, D., Lu, J.: GaussianFormer-2: Probabilistic gaussian superposition for efficient 3d occupancy prediction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 27477–27486 (2025)
work page 2025
-
[13]
In: European Conference on Computer Vision
Huang, Y., Zheng, W., Zhang, Y., Zhou, J., Lu, J.: GaussianFormer: Scene as gaus- sians for vision-based 3D semantic occupancy prediction. In: European Conference on Computer Vision. pp. 376–393 (2024)
work page 2024
-
[14]
In: Proceedings of the AAAI Conference on Artificial Intelligence
Jiang, X., Li, S., Liu, Y., Wang, S., Jia, F., Wang, T., Han, L., Zhang, X.: Far3D: Expanding the horizon for surround-view 3d object detection. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 2561–2569 (2024)
work page 2024
-
[15]
In: Proceedings of the IEEE International Conference on Robotics and Automation
Kim, J., Seong, M., Bang, G., Kum, D., Choi, J.W.: RCM-Fusion: Radar-camera multi-level fusion for 3d object detection. In: Proceedings of the IEEE International Conference on Robotics and Automation. pp. 18236–18242 (2024)
work page 2024
-
[16]
Advances in Neural Information Processing Systems37, 108625–108648 (2024)
Kim, J., Seong, M., Choi, J.W.: CRT-Fusion: Camera, radar, temporal fusion using motion information for 3d object detection. Advances in Neural Information Processing Systems37, 108625–108648 (2024)
work page 2024
-
[17]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Kim, Y., Shin, J., Kim, S., Lee, I.J., Choi, J.W., Kum, D.: CRN: Camera radar net for accurate, robust, efficient 3d perception. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 17615–17626 (2023)
work page 2023
-
[18]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops
Lee, Y., Hwang, J.w., Lee, S., Bae, Y., Park, J.: An energy and GPU-computation efficient backbone network for real-time object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 752–760 (2019) Horizon3D 17
work page 2019
-
[19]
In: Proceedings of the AAAI Conference on Artificial Intelligence
Li, Y., Yang, Y., Lei, Z.: RCTrans: Radar-camera transformer via radar densifier and sequential decoder for 3d object detection. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 39, pp. 5048–5056 (2025)
work page 2025
-
[20]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Li, Z., Lan, S., Alvarez, J.M., Wu, Z.: BEVNeXt: Reviving dense BEV frameworks for 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 20113–20123 (2024)
work page 2024
-
[21]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Lin, Z., Liu, Z., Xia, Z., Wang, X., Wang, Y., Qi, S., Dong, Y., Dong, N., Zhang, L., Zhu, C.: RCBEVDet: Radar-camera fusion in bird’s eye view for 3d object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14928–14937 (2024)
work page 2024
-
[22]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Liu, S., Cui, M., Li, B., Liang, Q., Hong, T., Huang, K., Shan, Y., Huang, K.: FSHNet: Fully sparse hybrid network for 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 8900–8909 (2025)
work page 2025
-
[23]
In: European Conference on Computer Vision
Liu, Y., Wang, T., Zhang, X., Sun, J.: PETR: Position embedding transformation for multi-view 3d object detection. In: European Conference on Computer Vision. pp. 531–548 (2022)
work page 2022
-
[24]
Advances in Neural Information Processing Systems37, 13601–13626 (2024)
Liu, Z., Hou, J., Wang, X., Ye, X., Wang, J., Zhao, H., Bai, X.: LION: Linear group RNN for 3D object detection in point clouds. Advances in Neural Information Processing Systems37, 13601–13626 (2024)
work page 2024
-
[25]
In: Pro- ceedings of the IEEE International Conference on Robotics and Automation
Liu, Z., Tang, H., Amini, A., Yang, X., Mao, H., Rus, D.L., Han, S.: BEVFusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation. In: Pro- ceedings of the IEEE International Conference on Robotics and Automation. pp. 2774–2781 (2023)
work page 2023
-
[26]
IEEE Robotics and Automation Letters7(2), 4961–4968 (2022)
Palffy, A., Pool, E., Baratam, S., Kooij, J.F., Gavrila, D.M.: Multi-class road user detection with 3+1D radar in the View-of-Delft dataset. IEEE Robotics and Automation Letters7(2), 4961–4968 (2022)
work page 2022
-
[27]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Palladin, E., Brucker, S., Ghilotti, F., Narayanan, P., Bijelic, M., Heide, F.: Self- supervised sparse sensor fusion for long range perception. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 27498–27509 (2025)
work page 2025
-
[28]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Wang, H., Shi, C., Shi, S., Lei, M., Wang, S., He, D., Schiele, B., Wang, L.: DSVT: Dynamic sparse voxel transformer with rotated sets. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13520–13529 (2023)
work page 2023
-
[29]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Wang, H., Gao, J., Hu, W., Zhang, Z.: Height-fidelity dense global fusion for multi-modal 3d object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 26664–26674 (2025)
work page 2025
-
[30]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops
Wang, T., Zhu, X., Pang, J., Lin, D.: FCOS3D: Fully convolutional one-stage monocular 3D object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops. pp. 913–922 (2021)
work page 2021
-
[31]
Wilson, B., Qi, W., Agarwal, T., Lambert, J., Singh, J., Khandelwal, S., Pan, B., Kumar, R., Hartnett, A., Pontes, J.K., Ramanan, D., Carr, P., Hays, J.: Argoverse 2: Next generation datasets for self-driving perception and forecasting. In: Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS Datasets and Benc...
work page 2021
-
[32]
In: Proceedings of the IEEE International Conference on Robotics and Automation
Wolters, P., Gilg, J., Teepe, T., Herzog, F., Fent, F., Rigoll, G.: SpaRC: Sparse radar- camera fusion for 3D object detection. In: Proceedings of the IEEE International Conference on Robotics and Automation. pp. 6589–6596 (2026)
work page 2026
-
[33]
Wolters, P., Gilg, J., Teepe, T., Herzog, F., Laouichi, A., Hofmann, M., Rigoll, G.: Unleashing HyDRa: Hybrid fusion, depth consistency and radar for unified 3d 18 G. Bang et al. perception. In: Proceedings of the IEEE International Conference on Robotics and Automation. pp. 7467–7474 (2025)
work page 2025
-
[34]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Xie, Y., Xu, C., Rakotosaona, M.J., Rim, P., Tombari, F., Keutzer, K., Tomizuka, M., Zhan, W.: SparseFusion: Fusing multi-modal sparse representations for multi-sensor 3d object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 17591–17602 (2023)
work page 2023
-
[35]
IEEE Transactions on Intelligent Vehicles9(1), 79–92 (2024)
Xiong, W., Liu, J., Huang, T., Han, Q.L., Xia, Y., Zhu, B.: LXL: Lidar excluded lean 3d object detection with 4d imaging radar and camera fusion. IEEE Transactions on Intelligent Vehicles9(1), 79–92 (2024)
work page 2024
-
[36]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Yang, Z., Yu, Z., Choy, C., Wang, R., Anandkumar, A., Alvarez, J.M.: Improving distant 3D object detection using 2D box supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14853– 14863 (2024)
work page 2024
-
[37]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Yin, T., Zhou, X., Krahenbuhl, P.: Center-based 3d object detection and tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11784–11793 (2021)
work page 2021
-
[38]
Advances in Neural Information Processing Systems36, 53076–53089 (2023)
Zhang,G.,Chen,J.,Gao,G.,Li,J.,Hu,X.:HEDNet:Ahierarchicalencoder-decoder network for 3d object detection in point clouds. Advances in Neural Information Processing Systems36, 53076–53089 (2023)
work page 2023
-
[39]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Zhang, G., Chen, J., Gao, G., Li, J., Liu, S., Hu, X.: SAFDNet: A simple and effective network for fully sparse 3d object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14477–14486 (2024)
work page 2024
-
[40]
Advances in Neural Information Processing Systems37, 81489–81509 (2024)
Zhang, G., Fan, L., He, C., Lei, Z., Zhang, Z., Zhang, L.: Voxel Mamba: Group-free state space models for point cloud based 3d object detection. Advances in Neural Information Processing Systems37, 81489–81509 (2024)
work page 2024
-
[41]
In: European Conference on Computer Vision
Zhang, H., Liang, L., Zeng, P., Song, X., Wang, Z.: SparseLIF: High-performance sparse lidar-camera fusion for 3d object detection. In: European Conference on Computer Vision. pp. 109–128 (2024)
work page 2024
-
[42]
In: International Conference on Learning Representations (2026)
Zhao, X., Liu, C., Zhu, M., Zhang, Z., Song, L., Luo, Q., Guo, C., Su, K.: Gaus- sianFusion: Unified 3D gaussian representation for multi-modal fusion perception. In: International Conference on Learning Representations (2026)
work page 2026
-
[43]
In: IEEE International Conference on Intelligent Transportation Systems
Zheng, L., Ma, Z., Zhu, X., Tan, B., Li, S., Long, K., Sun, W., Chen, S., Zhang, L., Wan, M., et al.: TJ4DRadSet: A 4d radar dataset for autonomous driving. In: IEEE International Conference on Intelligent Transportation Systems. pp. 493–498 (2022)
work page 2022
-
[44]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Zhong, H., Xiang, Z., Xu, R., Fu, J., Xu, P., Wang, S., Yang, Z., Pu, T., Liu, E.: CVFusion: Cross-view fusion of 4D radar and camera for 3D object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 28188–28197 (2025)
work page 2025
-
[45]
Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable DETR: Deformable transformers for end-to-end object detection. In: International Conference on Learn- ing Representations (2021) Horizon3D 19 Supplementary Material for Horizon3D: Sparse Radar-Camera Fusion for Long-Range 3D Perception in Autonomous Driving This supplementary material provides...
work page 2021
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.