Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Street Gaussians without 3D Object Tracker

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Street scenes reconstruct without 3D object trackers.

desk verdict A solid empirical extension of Street Gaussians/S3Gaussian that replaces 3D trackers with a 2D foundation model plus LiDAR/ICP and a scheduled motion loss, with strong Waymo-NOTR results but an under-tested claim of autonomous error correction. read the letter →

arxiv 2412.05548 v4 pith:LJAXGOAY submitted 2024-12-07 cs.CV

Ruida Zhang , Chengxi Li , Chenyangguang Zhang , Xingyu Liu , Haili Yuan , Yanyan Li , Xiangyang Ji , Gim Hee Lee This is my paper · ORCID
classification cs.CV
keywords streetscenereconstruction3DGaussianSplatting2DobjecttrackingLiDARpointcloudHexPlanemotionlearningnovelviewsynthesisautonomousdriving
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that dynamic street-scene reconstruction does not need 3D object trackers or ground-truth object poses. It builds vehicle trajectories by projecting 2D segmentation masks from a general-purpose 2D tracker onto LiDAR points and aligning the partial clouds frame-by-frame with ICP into a canonical model. Those trajectories supervise a HexPlane-based motion field only during early training; afterwards the field is free to correct pose errors and interpolate through missed detections. The authors report that this pipeline outperforms existing pose-based and self-supervised methods on Waymo-NOTR and KITTI without any tracking annotations. If correct, it removes the main annotation bottleneck for turning raw driving logs into editable, re-renderable 3D scenes.

What carries the argument

The load-bearing machinery is a two-stage trajectory-to-motion pipeline. First, 2D segmentation masks $M$ from a general-purpose tracker are re-projected onto LiDAR to obtain per-view object clouds $O_t^{(j)}$, and ICP aligns each successive partial cloud to an incrementally built canonical model $O$; overlap thresholds decide whether a frame updates the model (above 30%), is recorded as pose only (10-30%), or is discarded as a failed detection (below 10%). Second, dynamic Gaussians are deformed through a HexPlane feature grid: six learnable planes ($xy$, $xz$, $yz$, $xt$, $yt$, $zt$) encode spatial-temporal features, and decoders predict per-point motion $\Delta X_t$ and color change $\Delta C_t$. The motion loss $\mathcal{L}_{\text{motion}} = \mathrm{avg}_{X \in O} |\Delta X_t - (T_t X - X)|$ supervises the field with the tracker-derived trajectory for the first 40% of iterations, then is removed so the implicit field can smooth over and correct tracker errors. Isotropic Gaussian marbles keep dynamic points simple so all deformation is carried by motion and color, making the learned motion the main carrier of the argument.

What would settle it

Take a Waymo sequence with known vehicle poses, run the pipeline with the 2D tracker's masks, and compare the estimated trajectory $T_t$ against the ground-truth poses frame by frame. The central claim would be falsified if frames accepted by the 30%/10% overlap rule systematically carry translation errors above a few tens of centimeters, or if removing the motion loss after 40% of training fails to improve dynamic-region rendering on occluded frames compared with keeping trajectory supervision throughout.

Watch

Extended reading notes

Core claim

The central claim is that a 3D object tracker is unnecessary for high-quality dynamic street reconstruction. The paper's replacement is a stable object-tracking module that lifts 2D tracks to 3D, plus a motion-learning module that uses those tracks as soft supervision rather than rigid truth. The key move is to treat the trajectory-derived pose $T_t$ as a temporary teacher: the model learns per-point motion $\Delta X_t$ in HexPlane feature space, is pushed toward the teacher for the first 40% of training, and is then free to deviate, which lets it correct pose errors and recover missed detections. The paper reports that on Waymo-NOTR this beats Street Gaussians (3D-tracker-based) and S3Gaussian (self-supervised), and on KITTI it matches pose-supervised Street Gaussians on PSNR while exceeding it on SSIM and LPIPS.

Load-bearing premise

The load-bearing premise is that the 2D segmentation masks and the camera-LiDAR calibration are accurate enough that frame-by-frame ICP builds a clean canonical model of each vehicle; if masks are wrong or calibration is off, the trajectory loss teaches the motion field the wrong motion and the overlap thresholds cannot tell a real tracking failure from a bad alignment.

Editorial extensions

If this is right

  • New street scenes can be reconstructed from raw multi-view video plus LiDAR with no manual object poses and no 3D tracker, needing only a pretrained 2D tracker and sensor calibration.
  • Tracking errors no longer break rendering: frames discarded by the 10% overlap rule simply drop out of trajectory supervision, and the motion field can interpolate through them.
  • Because motion is per-Gaussian rather than a single rigid pose, within-object non-rigid changes such as open doors are representable.
  • Novel views can be rendered at arbitrary timestamps, including frames where the object was never successfully tracked, by querying HexPlane features at the desired time.
  • The method's performance is stable across two different 2D trackers (GLEE and UNINEXT), indicating the pipeline does not hinge on one specific tracker.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same 2D-mask-plus-ICP canonicalization could be applied to other dynamic classes such as pedestrians, where per-point deformation would handle articulated motion more naturally than rigid poses.
  • Editorial inference: the 40% supervision-removal schedule suggests a broader principle, that explicit pose priors work best as an initial curriculum for implicit motion fields, which could transfer to other dynamic reconstruction settings.
  • Editorial inference: because the method relies only on 2D tracking and LiDAR, it could in principle reconstruct scenes from any driving dataset with those sensors, even ones with no 3D annotations; the Waymo and KITTI experiments support but do not prove that.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes replacing 3D object trackers in dynamic street-scene Gaussian splatting with a pipeline built on a 2D foundation-model tracker. Vehicles are tracked in 2D with GLEE, lifted to 3D by projecting LiDAR points through segmentation masks, associated across cameras, and accumulated into a canonical point cloud via incremental ICP alignment, which also yields per-frame object poses. These poses are used to supervise per-point deformations predicted by a HexPlane network for the first 40% of training iterations, after which the motion loss is removed and photometric and geometric losses dominate. The method is evaluated on Waymo-NOTR and KITTI for scene reconstruction and novel view synthesis, with ablations targeting the tracking module, the motion loss, the isotropic marble representation, and the choice of 2D tracker.

Significance. The practical motivation is strong: avoiding 3D trackers and ground-truth object poses would broaden the applicability of dynamic street-scene reconstruction, and the use of 2D foundation models is a reasonable route toward better domain generalization. The paper also reports useful ablations, a comparison against rigid-transformation motion modeling, an analysis of tracking error distributions, and explicit failure cases in the supplementary material. If the central robustness claim were quantitatively established, the contribution would be significant. However, the headline claim of outperforming existing methods is not uniformly supported by the KITTI results, and the load-bearing claim that the HexPlane motion learning corrects trajectory errors and recovers missed detections is currently supported only indirectly.

major comments (3)
  1. [Abstract; §4.2, Table 2] The abstract and the contribution list state that the method "outperforms existing approaches" on Waymo-NOTR and KITTI, but Table 2 shows KITTI-75% PSNR 25.49 versus 25.79 for Street Gaussians (GT pose) and KITTI-50% PSNR 25.11 versus 25.52. The method wins on SSIM and LPIPS in both settings but loses on PSNR. Please qualify the claim, for example by saying "competitive on KITTI with improved SSIM/LPIPS" or by providing an aggregate or per-scene analysis that justifies a global superiority statement. As written, the empirical headline is overstated.
  2. [§3.4, Eq. (7); §4.3, Table 3] The central claim that motion learning "autonomously corrects trajectory errors and recovers missed detections" is not directly tested. Ablation B versus H removes the motion loss entirely, causing a 1.04 dB drop in DPSNR; this demonstrates the value of the initial motion prior, but it does not isolate the error-correction effect. Because Lmotion is active for the first 40% of iterations, any systematic error in Tt is imprinted into the HexPlane features before photometric loss can act. Please add a controlled experiment in which the estimated trajectory Tt is corrupted by known translation/rotation noise, or in which synthetic missed-detection gaps of varying length are introduced, and report final DPSNR as a function of the noise level or gap length. Please also report the fraction of frames rejected by the 10% overlap rule and the typical missed-detection statistics on the test scenes. Without such evidence, the "recovers missed detections" statement remains unsupported.
  3. [§3.3, Eq. (4)] The 30%/10% overlap thresholds and the 50-point cross-camera association rule are introduced without sensitivity analysis. A misclassified frame either corrupts the canonical model with a misaligned partial point cloud or discards a useful observation, and since the canonical model provides both the initial Gaussian positions and the trajectory Tt used by Eq. (7), these thresholds directly affect the downstream motion prior. Please report the distribution of τoverlap on the evaluated scenes and a sensitivity study over the threshold values, or otherwise justify these choices with data.
minor comments (5)
  1. [§3.3, Eq. (4)] The notation is ambiguous: after defining Tt = ICP(Ot, O), the overlap is written as O ∩ Tt^{-1}Ot. If Tt aligns Ot to O, the overlap should be O ∩ Tt Ot unless Tt is defined with the opposite convention. Please clarify the transformation direction.
  2. [Table 2] The caption contains a typo: "Street Guassians" should be "Street Gaussians".
  3. [Tables 1–3] No error bars or standard deviations are reported. Since the Waymo-NOTR dynamic32 subset has only 32 scenes, reporting per-scene variance or confidence intervals would strengthen the comparisons.
  4. [§3.5 and Supplementary §12] The depth loss in Eq. (8) is said to compare rendered depth with LiDAR-estimated depth, but the normalization distance d = 80 appears only in the supplementary Eq. (12). Please state this normalization in the main text for reproducibility.
  5. [Figure 2] The overview figure is dense and many labels are hard to read at normal print size. Please enlarge the pipeline labels or split the figure into two panels.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: trajectory is an intermediate supervision signal, and all reported metrics are measured on held-out images against external ground truth.

full rationale

The paper's load-bearing claim is that 2D-tracker + LiDAR + ICP trajectories plus HexPlane motion learning can replace 3D trackers for street-scene reconstruction. Tracing the derivation chain, the trajectory Tt from Eqs. 3-4 is not the final claim; it is an intermediate supervision signal. The motion loss in Eq. 7 encourages DeltaXt ~ TtX - X, but the paper explicitly applies this loss only for the first 40% of iterations and then removes it, allowing photometric and depth losses to refine the motion. All reported metrics (PSNR, SSIM, LPIPS, DPSNR, DSSIM) are computed on held-out time steps against actual images and GT 3D bounding boxes projected to 2D; none of these metrics is defined in terms of the fitted trajectory or the motion-loss target. The ablations in Table 3 also compare variants against these external metrics rather than against the training signal. The self-citations present in the reference list (e.g., the authors' pose-estimation papers) are background related-work citations and are not used to justify the central claim, nor is any uniqueness theorem imported from the authors' prior work. The robustness claim that motion learning 'autonomously corrects trajectory errors and recovers missed detections' is less directly tested than one might wish, since the main text does not include a controlled trajectory-noise or missed-detection-gap experiment and the supplementary only shows clipped error distributions. However, that is an evidence-strength or correctness-risk concern, not circularity: the paper does not define its output in terms of its input, no equation reduces to another by construction, and no fitted parameter is renamed as a prediction. The derivation is therefore self-contained with respect to the external evaluation protocol.

Assumptions & free parameters 5 free parameters · 7 assumptions · 0 invented entities

The central empirical claims rest on hand-chosen thresholds and training schedules, plus strong assumptions about calibration and 2D tracker reliability. No new physical entities are introduced. The most consequential choices are the ICP overlap thresholds and the 40% motion-loss schedule, because they directly control the trajectories that seed the motion field.

free parameters (5)
  • ICP overlap thresholds = 30% update, 10% discard
    Hand-chosen in Sec 3.3; determines which frames are treated as tracking failures and therefore shapes the trajectory entering Lmotion.
  • View association threshold = 50 shared LiDAR points
    Hand-chosen in Sec 3.3; links objects across cameras and is necessary for the 3D fusion.
  • Motion-loss schedule = first 40% of iterations
    Hand-chosen in Sec 3.4; the removal of Lmotion after 40% is the mechanism claimed to correct pose errors, but no sensitivity study is provided.
  • Loss weights in Eq. 8 = lambda_rgb=1, lambda_depth=1, lambda_ssim=0.1, lambda_tv=0.1, lambda_color-reg=0.01, lambda_motion=1
    Hand-chosen weights; no ablation or tuning procedure is reported.
  • Depth normalization distance = 80 meters
    Predefined in supplementary Eq. 12 for the depth loss; clips depth supervision at 80 meters.
assumptions (7)
  • standard math Differentiable 3D Gaussian splatting formulation (Eqs. 1-2)
    Adopted from 3DGS [37] as the rendering backbone; no new derivation is provided.
  • domain assumption Camera-LiDAR extrinsic and intrinsic calibration are correct for reprojection (Eq. 3)
    If calibration is off, segmented point clouds are contaminated; the paper only mentions 'slight misalignment' and outlier removal, without quantifying tolerance.
  • domain assumption GLEE 2D tracker provides sufficiently accurate and temporally consistent instance masks on unseen driving scenes
    Used in Sec 3.3 to seed all 3D trajectories; the paper presents tracking error histograms but no numeric calibration on the test sets.
  • domain assumption ICP converges to the correct relative pose between partial LiDAR point clouds of a vehicle
    Sec 3.3 builds the canonical model and poses T_t by ICP; partial, sparse, and non-rigid vehicle geometry can violate this.
  • ad hoc to paper The 30%/10% overlap rules correctly classify tracking failure vs success
    These thresholds are introduced in Sec 3.3 without validation; they control which frames supervise Lmotion.
  • ad hoc to paper Removing Lmotion after 40% of iterations lets HexPlane correct trajectory errors via photometric loss
    Sec 3.4 states this as the error-correction mechanism, but no direct trajectory-correction measurement is reported.
  • domain assumption Vehicle point clouds are rigid enough for the ICP canonical model, while per-point non-rigid motion is learned later
    Sec 3.3 assumes rigidity for ICP; Sec 3.4 allows non-rigid deformations such as doors opening. The paper does not analyze the interaction or failure mode.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Street Gaussians without 3D Object Tracker." pith.science (2026). https://pith.science/paper/LJAXGOAY

@misc{pith2026241205548,
  author       = {Pith},
  title        = {Pith review of: Street Gaussians without 3D Object Tracker},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LJAXGOAY}},
  note         = {Machine review of arXiv:2412.05548}
}
read the original abstract

Realistic scene reconstruction in driving scenarios poses significant challenges due to fast-moving objects. Most existing methods rely on labor-intensive manual labeling of object poses to reconstruct dynamic objects in canonical space and move them based on these poses during rendering. While some approaches attempt to use 3D object trackers to replace manual annotations, the limited generalization of 3D trackers -- caused by the scarcity of large-scale 3D datasets -- results in inferior reconstructions in real-world settings. In contrast, 2D foundation models demonstrate strong generalization capabilities. To eliminate the reliance on 3D trackers and enhance robustness across diverse environments, we propose a stable object tracking module by leveraging associations from 2D deep trackers within a 3D object fusion strategy. We address inevitable tracking errors by further introducing a motion learning strategy in an implicit feature space that autonomously corrects trajectory errors and recovers missed detections. Experimental results on Waymo-NOTR and KITTI show that our method outperforms existing approaches. Our code will be released on https://lolrudy.github.io/No3DTrackSG/.

Figures

Figures reproduced from arXiv: 2412.05548 by the authors.

Figure 1
Figure 1. Comparison of 3D tracker-based Street Gaussians [ [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our method. To overcome the limited generalization of 3D object trackers, we introduce a robust object tracking module based on a 2D object tracker [84]. We integrate 2D tracking with LiDAR data to segment the object’s point cloud and incrementally reconstruct the object model in canonical space frame-by-frame. The canonical model is used to estimate object pose Tt and serves as the initialization for Ga… view at source ↗
Figure 3
Figure 3. Qualitative comparison of novel view synthesis on NOTR dataset. Best viewed with zoom. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Editing demonstrations on Waymo-NOTR. 3D tracker PSNR↑ SSIM↑ LPIPS↓ DPSNR↑ DSSIM↑ Street Gaussians CasTrack [82, 83] 25.61 0.816 0.163 21.07 0.597 Street Gaussians VoxelNext [11] 26.98 0.838 0.149 24.62 0.742 [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Distributions of tracking errors. GT Ours Inaccurate human motion [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Failure cases distinguish between static and dynamic elements within a scene. • The proposed method requires per-scene optimization. A promising direction for future work is the development of a feed-forward approach for predicting generalizable 3D Gaussians. 3 [PITH_…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Learnable Infinite Taylor Gaussian for Dynamic View Rendering

    cs.CV 2024-12 reject novelty 3.0 of 10

    A dynamic Gaussian rendering method that wraps a third-order time polynomial around an SC-GS style skeleton and reports higher PSNR, though the 'infinite Taylor' framing is a fitted residual network.

Reference graph

Works this paper leans on

123 extracted references · 55 canonical work pages · cited by 1 Pith paper

  1. [1]

    Mip-nerf: A multiscale representation for anti-aliasing neu- ral radiance fields

    Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neu- ral radiance fields. In Proceedings of the IEEE/CVF inter- national conference on computer vision, pages 5855–5864,

  2. [2]

    Mip-nerf 360: Unbounded anti-aliased neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5470–5479, 2022

    2022

  3. [3]

    Zip-nerf: Anti-aliased grid-based neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti-aliased grid-based neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 19697–19705, 2023. 2

    2023

  4. [4]

    Multiple object tracking in recent times: A literature re- view

    Mk Bashar, Samia Islam, Kashifa Kawaakib Hussain, Md Bakhtiar Hasan, ABM Rahman, and Md Hasanul Kabir. Multiple object tracking in recent times: A literature re- view. arXiv preprint arXiv:2209.04796, 2022. 3

    arXiv 2022

  5. [5]

    Simple online and realtime tracking

    Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, and Ben Upcroft. Simple online and realtime tracking. In 2016 IEEE international conference on image processing (ICIP), pages 3464–3468. IEEE, 2016. 3

    2016

  6. [6]

    nuscenes: A mul- timodal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A mul- timodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020. 2, 3, 6

    2020

  7. [7]

    Hexplane: A fast representa- tion for dynamic scenes

    Ang Cao and Justin Johnson. Hexplane: A fast representa- tion for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 130–141, 2023. 2, 3, 4, 5

    2023

  8. [8]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nico- las Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. InEuropean conference on computer vision , pages 213–229. Springer,

Show all 123 references
  1. [9]

    D2-city: a large-scale dashcam video dataset of diverse traffic scenarios

    Zhengping Che, Guangyu Li, Tracy Li, Bo Jiang, Xuefeng Shi, Xinsheng Zhang, Ying Lu, Guobin Wu, Yan Liu, and Jieping Ye. D2-city: a large-scale dashcam video dataset of diverse traffic scenarios. arXiv preprint arXiv:1904.01975,

    1904 arXiv

  2. [10]

    Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering

    Yurui Chen, Chun Gu, Junzhe Jiang, Xiatian Zhu, and Li Zhang. Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering. arXiv preprint arXiv:2311.18561, 2023. 3

    2023 arXiv

  3. [11]

    V oxelnext: Fully sparse voxelnet for 3d object detection and tracking

    Yukang Chen, Jianhui Liu, Xiangyu Zhang, Xiaojuan Qi, and Jiaya Jia. V oxelnext: Fully sparse voxelnet for 3d object detection and tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 21674–21683, 2023. 1, 3, 6, 8, 2

    2023

  4. [12]

    Schwing, Alexander Kirillov, and Rohit Girdhar

    Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In CVPR, 2022. 2, 3, 4

    2022

  5. [13]

    Real-time trajectory planning for autonomous driving with gaussian process and incre- mental refinement

    Jie Cheng, Yingbing Chen, Qingwen Zhang, Lu Gan, Chengju Liu, and Ming Liu. Real-time trajectory planning for autonomous driving with gaussian process and incre- mental refinement. In 2022 International Conference on Robotics and Automation (ICRA), pages 8999–9005. IEEE,

    2022

  6. [14]

    Forecast-mae: Self-supervised pre-training for motion forecasting with masked autoencoders

    Jie Cheng, Xiaodong Mei, and Ming Liu. Forecast-mae: Self-supervised pre-training for motion forecasting with masked autoencoders. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 8679– 8689, 2023. 1

    2023

  7. [15]

    High-quality streamable free- viewpoint video

    Alvaro Collet, Ming Chuang, Pat Sweeney, Don Gillett, Dennis Evseev, David Calabrese, Hugues Hoppe, Adam Kirk, and Steve Sullivan. High-quality streamable free- viewpoint video. ACM Transactions on Graphics (ToG) , 34(4):1–13, 2015. 3

    2015

  8. [16]

    Parting with misconceptions about learning-based vehicle motion planning

    Daniel Dauner, Marcel Hallgarten, Andreas Geiger, and Kashyap Chitta. Parting with misconceptions about learning-based vehicle motion planning. In Conference on Robot Learning, pages 1268–1281. PMLR, 2023. 1

    2023

  9. [17]

    Imagenet: A large-scale hierarchical im- age database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical im- age database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 3

    2009

  10. [18]

    Gpv- pose: Category-level object pose estimation via geometry- guided point-wise voting

    Yan Di, Ruida Zhang, Zhiqiang Lou, Fabian Manhardt, Xi- angyang Ji, Nassir Navab, and Federico Tombari. Gpv- pose: Category-level object pose estimation via geometry- guided point-wise voting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2022

  11. [19]

    3dmotformer: Graph transformer for online 3d multi-object tracking

    Shuxiao Ding, Eike Rehder, Lukas Schneider, Marius Cordts, and Juergen Gall. 3dmotformer: Graph transformer for online 3d multi-object tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 9784–9794, 2023. 3

    2023

  12. [20]

    Carla: An open urban driv- ing simulator

    Alexey Dosovitskiy, German Ros, Felipe Codevilla, Anto- nio Lopez, and Vladlen Koltun. Carla: An open urban driv- ing simulator. InConference on robot learning, pages 1–16. PMLR, 2017. 3

    2017

  13. [21]

    An empirical study of the generalization ability of lidar 3d object detectors to unseen domains

    George Eskandar. An empirical study of the generalization ability of lidar 3d object detectors to unseen domains. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition , pages 23815–23825, 2024. 1, 3

    2024

  14. [22]

    9 Dynamic 3d gaussian fields for urban areas

    Tobias Fischer, Jonas Kulhanek, Samuel Rota Bulo, Lorenzo Porzi, Marc Pollefeys, and Peter Kontschieder. 9 Dynamic 3d gaussian fields for urban areas. arXiv preprint arXiv:2406.03175, 2024. 1

    2024 arXiv

  15. [23]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE conference on computer vision and pattern recognition, pages 3354–3361. IEEE, 2012. 2, 3, 6

    2012

  16. [24]

    Vision meets robotics: The kitti dataset.The Inter- national Journal of Robotics Research, 32(11):1231–1237,

    Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset.The Inter- national Journal of Robotics Research, 32(11):1231–1237,

  17. [25]

    Vip3d: End-to- end visual trajectory prediction via 3d agent queries

    Junru Gu, Chenxu Hu, Tianyuan Zhang, Xuanyao Chen, Yilun Wang, Yue Wang, and Hang Zhao. Vip3d: End-to- end visual trajectory prediction via 3d agent queries. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 5496–5506, 2023. 1

    2023

  18. [26]

    Robust non-rigid motion tracking and sur- face reconstruction using l0 regularization

    Kaiwen Guo, Feng Xu, Yangang Wang, Yebin Liu, and Qionghai Dai. Robust non-rigid motion tracking and sur- face reconstruction using l0 regularization. In Proceedings of the IEEE International Conference on Computer Vision, pages 3083–3091, 2015. 3

    2015

  19. [27]

    The re- lightables: V olumetric performance capture of humans with realistic relighting

    Kaiwen Guo, Peter Lincoln, Philip Davidson, Jay Busch, Xueming Yu, Matt Whalen, Geoff Harvey, Sergio Orts- Escolano, Rohit Pandey, Jason Dourgarian, et al. The re- lightables: V olumetric performance capture of humans with realistic relighting. ACM Transactions on Graphics (To...

    2019

  20. [28]

    Lvis: A dataset for large vocabulary instance segmentation

    Agrim Gupta, Piotr Dollar, and Ross Girshick. Lvis: A dataset for large vocabulary instance segmentation. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5356–5364, 2019. 2

    2019

  21. [29]

    St-p3: End-to-end vision- based autonomous driving via spatial-temporal feature learning

    Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, and Dacheng Tao. St-p3: End-to-end vision- based autonomous driving via spatial-temporal feature learning. In European Conference on Computer Vision , pages 533–549. Springer, 2022. 1

    2022

  22. [30]

    Hvtr: Hybrid volumetric-textural render- ing for human avatars

    Tao Hu, Tao Yu, Zerong Zheng, He Zhang, Yebin Liu, and Matthias Zwicker. Hvtr: Hybrid volumetric-textural render- ing for human avatars. In2022 International Conference on 3D Vision (3DV), pages 197–208. IEEE, 2022. 3

    2022

  23. [31]

    Planning-oriented autonomous driving

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17853–17862, 2023. 1

    2023

  24. [32]

    S3gaussian: Self-supervised street gaussians for autonomous driving

    Nan Huang, Xiaobao Wei, Wenzhao Zheng, Pengju An, Ming Lu, Wei Zhan, Masayoshi Tomizuka, Kurt Keutzer, and Shanghang Zhang. S3gaussian: Self-supervised street gaussians for autonomous driving. arXiv preprint arXiv:2405.20323, 2024. 2, 3, 5, 6, 7, 1

    2024 arXiv

  25. [33]

    A survey on trajectory-prediction methods for autonomous driving

    Yanjun Huang, Jiatong Du, Ziru Yang, Zewei Zhou, Lin Zhang, and Hong Chen. A survey on trajectory-prediction methods for autonomous driving. IEEE Transactions on Intelligent Vehicles, 7(3):652–674, 2022. 1

    2022

  26. [34]

    Givepose: Gradual intra- class variation elimination for rgb-based category-level ob- ject pose estimation

    Ziqin Huang, Gu Wang, Chenyangguang Zhang, Ruida Zhang, Xiu Li, and Xiangyang Ji. Givepose: Gradual intra- class variation elimination for rgb-based category-level ob- ject pose estimation. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 22055–22066,

  27. [35]

    Vad: Vectorized scene rep- resentation for efficient autonomous driving

    Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jia- jie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene rep- resentation for efficient autonomous driving. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vi...

    2023

  28. [36]

    Analysis based on recent deep learning approaches applied in real-time multi-object tracking: a review

    Lesole Kalake, Wanggen Wan, and Li Hou. Analysis based on recent deep learning approaches applied in real-time multi-object tracking: a review. IEEE Access , 9:32650– 32671, 2021. 3

    2021

  29. [37]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  30. [38]

    Autosplat: Constrained gaussian splatting for au- tonomous driving scene reconstruction

    Mustafa Khan, Hamidreza Fazlali, Dhruv Sharma, Tong- tong Cao, Dongfeng Bai, Yuan Ren, and Bingbing Liu. Autosplat: Constrained gaussian splatting for au- tonomous driving scene reconstruction. arXiv preprint arXiv:2407.02598, 2024. 1, 3, 6

    2024 arXiv

  31. [39]

    Adam: A method for stochastic opti- mization

    Diederik P Kingma. Adam: A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980, 2014. 6

    2014 arXiv

  32. [40]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4015–4026, 2023. 2, 3

    2023

  33. [41]

    The open images dataset v4: Unified image classifica- tion, object detection, and visual relationship detection at scale

    Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Ui- jlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Ste- fan Popov, Matteo Malloci, Alexander Kolesnikov, et al. The open images dataset v4: Unified image classifica- tion, object detection, and visual relationship detec...

    1956

  34. [42]

    Pillarnext: Re- thinking network designs for 3d object detection in lidar point clouds

    Jinyu Li, Chenxu Luo, and Xiaodong Yang. Pillarnext: Re- thinking network designs for 3d object detection in lidar point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17567– 17576, 2023. 1, 3

    2023

  35. [43]

    Aads: Aug- mented autonomous driving simulation using data-driven algorithms

    Wei Li, CW Pan, Rong Zhang, JP Ren, YX Ma, Jin Fang, FL Yan, QC Geng, XY Huang, HJ Gong, et al. Aads: Aug- mented autonomous driving simulation using data-driven algorithms. Science robotics, 4(28):eaaw0863, 2019. 3

    2019

  36. [44]

    Geogaussian: Geometry-aware gaussian splatting for scene rendering

    Yanyan Li, Chenyu Lyu, Yan Di, Guangyao Zhai, Gim Hee Lee, and Federico Tombari. Geogaussian: Geometry-aware gaussian splatting for scene rendering. InEuropean Confer- ence on Computer Vision, pages 441–457. Springer, 2024. 2

    2024

  37. [45]

    Ro- bust 3d human motion reconstruction via dynamic template construction

    Zhong Li, Yu Ji, Wei Yang, Jinwei Ye, and Jingyi Yu. Ro- bust 3d human motion reconstruction via dynamic template construction. In 2017 International Conference on 3D Vi- sion (3DV), pages 496–505. IEEE, 2017. 3

    2017

  38. [46]

    Zhiqi Li, Zhiding Yu, Shiyi Lan, Jiahan Li, Jan Kautz, Tong Lu, and Jose M Alvarez. Is ego status all you need for open- loop end-to-end autonomous driving? In Proceedings of 10 the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14864–14873, 2024. 1

    2024

  39. [47]

    Pnpnet: End-to-end per- ception and prediction with tracking in the loop

    Ming Liang, Bin Yang, Wenyuan Zeng, Yun Chen, Rui Hu, Sergio Casas, and Raquel Urtasun. Pnpnet: End-to-end per- ception and prediction with tracking in the loop. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11553–11562, 2020. 1

    2020

  40. [48]

    Gaufre: Gaussian deformation fields for real-time dynamic novel view synthesis

    Yiqing Liang, Numair Khan, Zhengqin Li, Thu Nguyen- Phuoc, Douglas Lanman, James Tompkin, and Lei Xiao. Gaufre: Gaussian deformation fields for real-time dynamic novel view synthesis. arXiv preprint arXiv:2312.11458 ,

    arXiv

  41. [49]

    Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d

    Yiyi Liao, Jun Xie, and Andreas Geiger. Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 45(3):3292–3310, 2022. 2, 3

    2022

  42. [50]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedin...

    2014

  43. [51]

    CATRE: iterative point clouds alignment for category-level object pose refinement

    Xingyu Liu, Gu Wang, Yi Li, and Xiangyang Ji. CATRE: iterative point clouds alignment for category-level object pose refinement. In European Conference on Computer Vi- sion, 2022. 3

    2022

  44. [52]

    Rasim: A range-aware high- fidelity rgb-d data simulation pipeline for real-world appli- cations

    Xingyu Liu, Chenyangguang Zhang, Gu Wang, Ruida Zhang, and Xiangyang Ji. Rasim: A range-aware high- fidelity rgb-d data simulation pipeline for real-world appli- cations. In 2024 IEEE international conference on robotics and automation (ICRA), pages 17057–17064. IEEE, 2024. 3

    2024

  45. [53]

    Gfreedet: Exploiting gaussian splatting and foundation models for model-free unseen object detection in the bop challenge

    Xingyu Liu, Yingyue Li, Chengxi Li, Gu Wang, Chenyang- guang Zhang, Ziqin Huang, and Xiangyang Ji. Gfreedet: Exploiting gaussian splatting and foundation models for model-free unseen object detection in the bop challenge

  46. [54]

    UNOPose: Unseen object pose estimation with an unposed rgb-d refer- ence image

    Xingyu Liu, Gu Wang, Ruida Zhang, Chenyangguang Zhang, Federico Tombari, and Xiangyang Ji. UNOPose: Unseen object pose estimation with an unposed rgb-d refer- ence image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 3

    2025

  47. [55]

    Gdrnpp: A geometry-guided and fully learning-based object pose es- timator

    Xingyu Liu, Ruida Zhang, Chenyangguang Zhang, Gu Wang, Jiwen Tang, Zhigang Li, and Xiangyang Ji. Gdrnpp: A geometry-guided and fully learning-based object pose es- timator. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 2025. 3

    2025

  48. [56]

    Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation

    Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela L Rus, and Song Han. Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation. In 2023 IEEE international conference on robotics and automation (ICRA), pages 2774–2781. IEEE,

    2023

  49. [57]

    Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. arXiv preprint arXiv:2308.09713, 2023. 3

    2023 arXiv

  50. [58]

    Multiple object tracking: A literature review

    Wenhan Luo, Junliang Xing, Anton Milan, Xiaoqin Zhang, Wei Liu, and Tae-Kyun Kim. Multiple object tracking: A literature review. Artificial intelligence, 293:103448, 2021. 3

    2021

  51. [59]

    One million scenes for autonomous driving: Once dataset

    Jiageng Mao, Minzhe Niu, Chenhan Jiang, Hanxue Liang, Jingheng Chen, Xiaodan Liang, Yamin Li, Chaoqiang Ye, Wei Zhang, Zhenguo Li, et al. One million scenes for autonomous driving: Once dataset. arXiv preprint arXiv:2106.11037, 2021. 3

    2021 arXiv

  52. [60]

    3d object detection for autonomous driving: A comprehensive survey

    Jiageng Mao, Shaoshuai Shi, Xiaogang Wang, and Hong- sheng Li. 3d object detection for autonomous driving: A comprehensive survey. International Journal of Computer Vision, 131(8):1909–1963, 2023. 3

    1909

  53. [61]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM, 65(1):99–106, 2021. 2, 3

    2021

  54. [62]

    Simple open-vocabulary object de- tection with vision transformers

    M Minderer, A Gritsenko, A Stone, M Neumann, D Weis- senborn, A Dosovitskiy, A Mahendran, A Arnab, M De- hghani, Z Shen, et al. Simple open-vocabulary object de- tection with vision transformers. arxiv 2022. arXiv preprint arXiv:2205.06230, 2, 2022. 2

    2022 arXiv

  55. [63]

    Instant neural graphics primitives with a multiresolution hash encoding

    Thomas M ¨uller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1–15, 2022. 2

    2022

  56. [64]

    Dinov2: Learning robust visual features without supervi- sion

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervi- sion. arXiv preprint arXiv:2304.07193, 2023. 2, 3

    2023 arXiv

  57. [65]

    Neural scene graphs for dynamic scenes

    Julian Ost, Fahim Mannan, Nils Thuerey, Julian Knodt, and Felix Heide. Neural scene graphs for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 2856–2865, 2021. 1, 3, 6, 8

    2021

  58. [66]

    Simpletrack: Understanding and rethinking 3d multi-object tracking

    Ziqi Pang, Zhichao Li, and Naiyan Wang. Simpletrack: Understanding and rethinking 3d multi-object tracking. In European Conference on Computer Vision, pages 680–696. Springer, 2022. 3, 6, 8

    2022

  59. [67]

    Desire-gs: 4d street gaussians for static-dynamic decomposition and surface re- construction for urban driving scenes

    Chensheng Peng, Chengwei Zhang, Yixiao Wang, Chen- feng Xu, Yichen Xie, Wenzhao Zheng, Kurt Keutzer, Masayoshi Tomizuka, and Wei Zhan. Desire-gs: 4d street gaussians for static-dynamic decomposition and surface re- construction for urban driving scenes. In Proceedings of the C...

    2025

  60. [68]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International conference on machine learning...

    2021

  61. [69]

    Domain generalization of 3d semantic seg- mentation in autonomous driving

    Jules Sanchez, Jean-Emmanuel Deschaud, and Franc ¸ois Goulette. Domain generalization of 3d semantic seg- mentation in autonomous driving. In Proceedings of the 11 IEEE/CVF International Conference on Computer Vision , pages 18077–18087, 2023. 3

    2023

  62. [70]

    K- planes: Explicit radiance fields in space, time, and appear- ance

    Sara Fridovich-Keil and Giacomo Meanti, Frederik Rah- bæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K- planes: Explicit radiance fields in space, time, and appear- ance. In CVPR, 2023. 5, 2

    2023

  63. [71]

    Airsim: High-fidelity visual and physical sim- ulation for autonomous vehicles

    Shital Shah, Debadeepta Dey, Chris Lovett, and Ashish Kapoor. Airsim: High-fidelity visual and physical sim- ulation for autonomous vehicles. In Field and Service Robotics: Results of the 11th International Conference , pages 621–635. Springer, 2018. 3

    2018

  64. [72]

    Towards efficient neural scene graphs by learning consistency fields

    Yeji Song, Chaerin Kong, Seoyoung Lee, Nojun Kwak, and Joonseok Lee. Towards efficient neural scene graphs by learning consistency fields. arXiv preprint arXiv:2210.04127, 2022. 3

    2022 arXiv

  65. [73]

    Mdt3d: Multi-dataset training for lidar 3d object detection generalization

    Louis Soum-Fontez, Jean-Emmanuel Deschaud, and Franc ¸ois Goulette. Mdt3d: Multi-dataset training for lidar 3d object detection generalization. In 2023 IEEE/RSJ In- ternational Conference on Intelligent Robots and Systems (IROS), pages 5765–5772. IEEE, 2023. 1, 3

    2023

  66. [74]

    Dynamic gaussian marbles for novel view synthesis of casual monocular videos

    Colton Stearns, Adam Harley, Mikaela Uy, Florian Du- bost, Federico Tombari, Gordon Wetzstein, and Leonidas Guibas. Dynamic gaussian marbles for novel view synthesis of casual monocular videos. arXiv preprint arXiv:2406.18717, 2024. 3, 4, 5

    2024 arXiv

  67. [75]

    Robustfusion: Human volumetric capture with data-driven visual cues using a rgbd camera

    Zhuo Su, Lan Xu, Zerong Zheng, Tao Yu, Yebin Liu, and Lu Fang. Robustfusion: Human volumetric capture with data-driven visual cues using a rgbd camera. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16 , pages 246–...

    2020

  68. [76]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aure- lien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF conference on computer ...

    2020

  69. [77]

    Lidarf: Delving into lidar for neural radiance field on street scenes

    Shanlin Sun, Bingbing Zhuang, Ziyu Jiang, Buyu Liu, Xi- aohui Xie, and Manmohan Chandraker. Lidarf: Delving into lidar for neural radiance field on street scenes. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19563–19572, 2024. 3

    2024

  70. [78]

    Suds: Scalable urban dynamic scenes

    Haithem Turki, Jason Y Zhang, Francesco Ferroni, and Deva Ramanan. Suds: Scalable urban dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition , pages 12375–12385, 2023. 3

    2023

  71. [79]

    Train in germany, test in the usa: Making 3d object detectors generalize

    Yan Wang, Xiangyu Chen, Yurong You, Li Erran Li, Bharath Hariharan, Mark Campbell, Kilian Q Weinberger, and Wei-Lun Chao. Train in germany, test in the usa: Making 3d object detectors generalize. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognit...

    2020

  72. [80]

    A comprehensive review of 3d object detection in au- tonomous driving: Technological advances and future di- rections

    Yu Wang, Shaohua Wang, Yicheng Li, and Mingchun Liu. A comprehensive review of 3d object detection in au- tonomous driving: Technological advances and future di- rections. arXiv preprint arXiv:2408.16530, 2024. 3

    2024 arXiv

  73. [81]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xi- aopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xing- gang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages...

    2024

  74. [82]

    3d multi-object tracking in point clouds based on prediction confidence-guided data association.IEEE Trans- actions on Intelligent Transportation Systems, 23(6):5668– 5677, 2021

    Hai Wu, Wenkai Han, Chenglu Wen, Xin Li, and Cheng Wang. 3d multi-object tracking in point clouds based on prediction confidence-guided data association.IEEE Trans- actions on Intelligent Transportation Systems, 23(6):5668– 5677, 2021. 2, 1

    2021

  75. [83]

    Casa: A cascade attention network for 3-d object detection from lidar point clouds.IEEE Transactions on Geoscience and Remote Sensing, 60:1–11, 2022

    Hai Wu, Jinhao Deng, Chenglu Wen, Xin Li, Cheng Wang, and Jonathan Li. Casa: A cascade attention network for 3-d object detection from lidar point clouds.IEEE Transactions on Geoscience and Remote Sensing, 60:1–11, 2022. 2, 1

    2022

  76. [84]

    General object foundation model for images and videos at scale

    Junfeng Wu, Yi Jiang, Qihao Liu, Zehuan Yuan, Xiang Bai, and Song Bai. General object foundation model for images and videos at scale. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 3783–3795, 2024. 1, 2, 3, 4, 6, 8

    2024

  77. [85]

    Mars: An instance-aware, modu- lar and realistic simulator for autonomous driving

    Zirui Wu, Tianyu Liu, Liyi Luo, Zhide Zhong, Jianteng Chen, Hongmin Xiao, Chao Hou, Haozhe Lou, Yuantao Chen, Runyi Yang, et al. Mars: An instance-aware, modu- lar and realistic simulator for autonomous driving. In CAAI International Conference on Artificial Intelligence , pag...

    2023

  78. [86]

    Pandaset: Advanced sensor suite dataset for au- tonomous driving

    Pengchuan Xiao, Zhenlei Shao, Steven Hao, Zishuo Zhang, Xiaolin Chai, Judy Jiao, Zesong Li, Jian Wu, Kai Sun, Kun Jiang, et al. Pandaset: Advanced sensor suite dataset for au- tonomous driving. In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) , p...

    2021

  79. [87]

    S-nerf: Neural radiance fields for street views

    Ziyang Xie, Junge Zhang, Wenye Li, Feihu Zhang, and Li Zhang. S-nerf: Neural radiance fields for street views. arXiv preprint arXiv:2303.00749, 2023. 3

    2023 arXiv

  80. [88]

    ViTPose: Simple vision transformer baselines for human pose estimation

    Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. ViTPose: Simple vision transformer baselines for human pose estimation. In Advances in Neural Information Pro- cessing Systems, 2022. 2

    2022

  81. [89]

    Vitpose+: Vision transformer foundation model for generic body pose estimation

    Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. Vitpose+: Vision transformer foundation model for generic body pose estimation. arXiv preprint arXiv:2212.04246 ,

    arXiv

  82. [90]

    Universal instance percep- tion as object discovery and retrieval

    Bin Yan, Yi Jiang, Jiannan Wu, Dong Wang, Ping Luo, Ze- huan Yuan, and Huchuan Lu. Universal instance percep- tion as object discovery and retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15325–15336, 2023. 1, 8

    2023

  83. [91]

    Street gaussians: Modeling dynamic urban scenes with gaussian splatting

    Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street gaussians: Modeling dynamic urban scenes with gaussian splatting. In ECCV, 2024. 1, 2, 3, 6, 7, 8

    2024

  84. [92]

    Emernerf: Emergent spatial- temporal scene decomposition via self-supervision

    Jiawei Yang, Boris Ivanovic, Or Litany, Xinshuo Weng, Seung Wook Kim, Boyi Li, Tong Che, Danfei Xu, Sanja Fidler, Marco Pavone, et al. Emernerf: Emergent spatial- temporal scene decomposition via self-supervision. arXiv preprint arXiv:2311.02077, 2023. 3, 6, 7, 1 12

    2023 arXiv

  85. [93]

    Unisim: A neural closed-loop sensor simulator

    Ze Yang, Yun Chen, Jingkang Wang, Sivabalan Mani- vasagam, Wei-Chiu Ma, Anqi Joyce Yang, and Raquel Ur- tasun. Unisim: A neural closed-loop sensor simulator. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 1389–1399, 2023. 3

    2023

  86. [94]

    Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting

    Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting. arXiv preprint arXiv:2310.10642, 2023. 3

    2023 arXiv

  87. [95]

    Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition , pages 20331–20341, 2024. 3

    2024

  88. [96]

    Center-based 3d object detection and tracking

    Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center-based 3d object detection and tracking. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11784–11793, 2021. 1, 3

    2021

  89. [97]

    Bdd100k: A diverse driving video database with scalable annotation tooling

    Fisher Yu, Wenqi Xian, Yingying Chen, Fangchen Liu, Mike Liao, Vashisht Madhavan, Trevor Darrell, et al. Bdd100k: A diverse driving video database with scalable annotation tooling. arXiv preprint arXiv:1805.04687, 2(5): 6, 2018. 2, 3

    2018 arXiv

  90. [98]

    Rethinking the open-loop evalua- tion of end-to-end autonomous driving in nuscenes

    Jiang-Tian Zhai, Ze Feng, Jinhao Du, Yongqiang Mao, Jiang-Jiang Liu, Zichang Tan, Yifu Zhang, Xiaoqing Ye, and Jingdong Wang. Rethinking the open-loop evalua- tion of end-to-end autonomous driving in nuscenes. arXiv preprint arXiv:2305.10430, 2023. 1

    2023 arXiv

  91. [99]

    Uni3d: A unified baseline for multi-dataset 3d object detection

    Bo Zhang, Jiakang Yuan, Botian Shi, Tao Chen, Yikang Li, and Yu Qiao. Uni3d: A unified baseline for multi-dataset 3d object detection. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 9253–9262, 2023. 1, 3

    2023

  92. [100]

    3d multiple object tracking on autonomous driving: A literature review

    Peng Zhang, Xin Li, Liang He, and Xin Lin. 3d multiple object tracking on autonomous driving: A literature review. arXiv preprint arXiv:2309.15411, 2023. 3

    2023 arXiv

  93. [101]

    Rbp-pose: Resid- ual bounding box projection for category-level pose esti- mation

    Ruida Zhang, Yan Di, Zhiqiang Lou, Fabian Manhardt, Federico Tombari, and Xiangyang Ji. Rbp-pose: Resid- ual bounding box projection for category-level pose esti- mation. In European conference on computer vision, pages 655–672. Springer, 2022. 3

    2022

  94. [102]

    Ssp-pose: Symmetry-aware shape prior deformation for direct category-level object pose estima- tion

    Ruida Zhang, Yan Di, Fabian Manhardt, Federico Tombari, and Xiangyang Ji. Ssp-pose: Symmetry-aware shape prior deformation for direct category-level object pose estima- tion. In 2022 IEEE/RSJ International Conference on Intel- ligent Robots and Systems (IROS), pages 7452–7459....

    2022

  95. [103]

    Lapose: Laplacian mixture shape modeling for rgb- based category-level object pose estimation

    Ruida Zhang, Ziqin Huang, Gu Wang, Chenyangguang Zhang, Yan Di, Xingxing Zuo, Jiwen Tang, and Xiangyang Ji. Lapose: Laplacian mixture shape modeling for rgb- based category-level object pose estimation. In European Conference on Computer Vision, pages 467–484. Springer,

  96. [104]

    Kp-red: Exploiting semantic keypoints for joint 3d shape retrieval and deformation

    Ruida Zhang, Chenyangguang Zhang, Yan Di, Fabian Man- hardt, Xingyu Liu, Federico Tombari, and Xiangyang Ji. Kp-red: Exploiting semantic keypoints for joint 3d shape retrieval and deformation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2024

  97. [105]

    Towards do- main generalization in object detection

    Xingxuan Zhang, Zekai Xu, Renzhe Xu, Jiashuo Liu, Peng Cui, Weitao Wan, Chong Sun, and Chen Li. Towards do- main generalization in object detection. arXiv preprint arXiv:2203.14387, 2022. 2

    2022 arXiv

  98. [106]

    Fairmot: On the fairness of detection and re-identification in multiple object tracking

    Yifu Zhang, Chunyu Wang, Xinggang Wang, Wenjun Zeng, and Wenyu Liu. Fairmot: On the fairness of detection and re-identification in multiple object tracking. International journal of computer vision, 129:3069–3087, 2021. 3

    2021

  99. [107]

    Hugs: Holistic urban 3d scene understanding via gaussian splatting

    Hongyu Zhou, Jiahao Shao, Lu Xu, Dongfeng Bai, We- ichao Qiu, Bingbing Liu, Yue Wang, Andreas Geiger, and Yiyi Liao. Hugs: Holistic urban 3d scene understanding via gaussian splatting. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, page...

    2024

  100. [108]

    Drivinggaussian: Composite gaussian splatting for surrounding dynamic au- tonomous driving scenes

    Xiaoyu Zhou, Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang. Drivinggaussian: Composite gaussian splatting for surrounding dynamic au- tonomous driving scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages...

    2024

  101. [110]

    As shown in Table 4, while our primary focus is on handling dynamic objects, our method also demonstrates robust per- formance in static scenes

    Evaluation on Static Scenes in Waymo- NOTR Dataset We evaluate our method on the static32 subset of the Waymo-NOTR dataset [76, 92], following the experimen- tal setup of EmerNeRF [92] for novel view synthesis. As shown in Table 4, while our primary focus is on handling dynami...

  102. [111]

    Method 3DGS [37] S3G [32] SG [91] Ours Speed (FPS) 200 15 160 100 Table 5

    Runtime Analysis As shown in Table 5, we evaluate the inference speed of our method and several state-of-the-art methods at a resolution of 960 × 640 on the same device. Method 3DGS [37] S3G [32] SG [91] Ours Speed (FPS) 200 15 160 100 Table 5. Inference speed at 960 × 640. S3...

  103. [112]

    Editing Examples We provide editing demonstrations in Fig. 4. Gaussians corresponding to cars are associated at initialization and consistently maintained throughout the optimization pro- cess. This enables object editing by directly applying rigid transformations to the corre...

  104. [113]

    Since pretrained nuScenes weights are unavailable, we instead use weights pretrained on KITTI

    Choice of 3D tracker To further illustrate the generalization challenges of 3D trackers, we employ CasTrack [82, 83] as the 3D tracker for Street Gaussians [91], using the same detection and track- ing algorithm as in the original paper. Since pretrained nuScenes weights are u...

  105. [114]

    Our 2D tracker-based method is more robust

    Tracking Errors Analysis We evaluate 3D trajectories from 2D and 3D trackers on Waymo-NOTR, measuring translation (Euclidean) and ro- tation errors (clipped at 1m or 30◦ ; with missing detections treated as max error). Our 2D tracker-based method is more robust. Error distribu...

  106. [115]

    Comparison with Rigid-transformation- based Motion Modeling On Waymo-NOTR, we use 3D trajectories computed from 2D tracking to model vehicle motion via rigid transfor- mations, following Street Gaussians (SG) [91] (see Tab. 7). While improved trajectories help, SG still underp...

  107. [116]

    (9) The components of the loss function are detailed below:

    Details of Loss Functions As described in the main paper, the total loss function is expressed as: L =λrgbLrgb + λssimLssim + λdepthLdepth + λtvLtv + λcolor-regLcolor-reg + λmotionLmotion. (9) The components of the loss function are detailed below:

  108. [117]

    Photometric L1 Loss (Lrgb): This L1 loss measures the photometric difference between the rendered image and the ground truth: Lrgb = ||Irender − Igt||1, (10) where Irender and Igt represent the rendered and ground truth images, respectively

  109. [118]

    (11) 1 Original Image MoveRemove Substitute Figure 4

    Structural Similarity Loss ( Lssim): This loss evaluates the structural similarity between Irender and Igt: Lssim = 1.0 − SSIM(Irender, Igt). (11) 1 Original Image MoveRemove Substitute Figure 4. Editing demonstrations on Waymo-NOTR. 3D tracker PSNR↑ SSIM↑ LPIPS↓ DPSNR↑ DSSIM↑...

  110. [119]

    Depth loss is calculated only for pix- els with ground truth depth values between 0.01 and 80 meters

    Depth Loss ( Ldepth): This L1 loss computes the differ- ence between the rendered depth map Drender and the ground truth depth map derived from LiDAR data Dgt: Ldepth = 1 d ||Drender − Dgt||1, (12) where d = 80 is the predefined maximum depth used for normalization. Depth loss...

  111. [120]

    Total Variation Loss (Ltv): A grid-based total variation loss is employed to encourage smooth gradients for Hex- Plane feature grids, following K-Planes [70]: Ltv = avgc,i,j ||P i,j c − P i−1,j c ||2 2 + ||P i,j c − P i,j−1 c ||2 2 , (13) where avg denotes the average operator...

  112. [121]

    Color Regularization Loss ( Lcolor-reg): This L1 regu- larization loss minimizes the predicted color change ∆C for each point to regularize the deformation network: Lcolor-reg = Σ||∆C||1. (14)

  113. [122]

    The weights assigned to each loss component are:λrgb = 1.0, λssim = 0.1, λdepth = 1.0, λtv = 0.1, λcolor-reg = 0.01, and λmotion = 1.0

    Motion Loss Lmotion: The motion loss is introduced in the main paper as, Lmotion = avgX ∈O|∆Xt − (TtX − X)|, (15) where X is the center position of a Gaussian in objectO. The weights assigned to each loss component are:λrgb = 1.0, λssim = 0.1, λdepth = 1.0, λtv = 0.1, λcolor-r...

  114. [123]

    This leads to inaccurate hu- man motion in some cases, which can be improved by incorporating human pose estimation as prior informa- tion (see Fig

    Limitation and Failure Cases • Our approach primarily focuses on modeling moving ve- hicles while using 4DGS [81] to model humans without explicit motion guidance. This leads to inaccurate hu- man motion in some cases, which can be improved by incorporating human pose estimati...

  115. [2024]

    IEEE/CVF Conference on Computer Vision and Pat- tern Recognition Workshop, 2025. 3

    2025

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.