Pith. sign in

REVIEW 4 major objections 6 minor 67 references

GS-LIVO: Real-Time LiDAR, Inertial, and Visual Multi-sensor Fused Odometry with Gaussian Mapping

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A LiDAR-inertial-visual odometry system keeps 3D Gaussian map updates running in real time on resource-constrained embedded hardware.

desk verdict Real-time Gaussian mapping on an embedded GPU is a genuinely useful integration, but the paper's own evidence undercuts its headline numbers and the global-consistency claim is untested on revisits. read the letter →

arxiv 2501.08672 v1 pith:YWTFVGZX submitted 2025-01-15 cs.RO cs.CV

Sheng Hong , Chunran Zheng , Yishu Shen , Changze Li , Fu Zhang , Tong Qin , Shaojie Shen This is my paper · ORCID
classification cs.ROcs.CV
keywords 3DGaussianSplattingSLAMLiDAR-inertial-visualodometrymulti-sensorfusionslidingwindowoptimizationembeddedsystemsoctreephotometricrendering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GS-LIVO tries to show that 3D Gaussian Splatting, a photorealistic scene representation that normally demands large GPU budgets, can be embedded in a tightly coupled LiDAR-inertial-visual odometry pipeline that runs in real time on a resource-constrained robot computer. The central claim is that by storing the global Gaussian map in CPU memory and optimizing only a sliding window of Gaussians inside the current field of view, map updates can be kept at over 10 Hz indoors and about 3 Hz outdoors while odometry accuracy stays competitive with established multi-sensor SLAM systems. The paper reports per-frame pipeline latencies below 90 ms on a desktop and about 48 ms on a small ARM-based embedded computer, with GPU memory use between 1.2 GB and 9.5 GB depending on scene scale. If the claim holds, photorealistic dense maps become usable online for navigation and planning rather than only for offline reconstruction.

What carries the argument

The load-bearing mechanism is the sliding window of Gaussians: a contiguous CPU buffer holding the Gaussians of the current field of view, mirrored on the GPU and updated incrementally frame to frame through a spatial hash table. When a voxel leaves the field of view, its optimized parameters are copied back into the global hash-indexed octree in RAM; when a new voxel enters, it is appended to the contiguous buffer and transferred to the GPU. This keeps host-to-device transfers proportional to newly visible voxels rather than to the entire map, and lets the Adam-based photometric optimization operate only on visible Gaussians. A second mechanism is the hash-indexed octree itself, which indexes the global map by spatial hash keys over sparse volumes and supports multiple voxel levels so that detail can adapt to indoor and outdoor scene scales.

What would settle it

Run the system on a trajectory that leaves a mapped area, travels away, and later returns: if the rendered old Gaussians visibly disagree with the new camera image, or if the odometry jumps when re-observing known structure, then sliding-window-only optimization is not sufficient for global consistency.

Watch

Extended reading notes

Core claim

The paper's discovery is that the practical bottleneck in Gaussian-based SLAM is the scope of map optimization, not the Gaussian representation itself. Restricting photometric optimization to a sliding window of currently visible Gaussians, while persisting all retired Gaussians in a hash-indexed octree in CPU memory, keeps per-frame GPU work and memory nearly constant as the map grows. The visual measurement model renders the current Gaussian window at the LiDAR-inertial predicted pose, compares the rendered image with the real camera image, and feeds the photometric residual and its Jacobians into an iterative error-state Kalman filter that also ingests LiDAR point-to-plane residuals and IMU propagation. The paper reports indoor trajectory RMSE of about 5 mm, outdoor aerial RMSE of 0.58 m, and a total pipeline time of 48.3 ms on the embedded platform, and presents the system as the first real-time Gaussian-based SLAM with online map updates on such hardware.

Load-bearing premise

The argument stands or falls on the assumption that optimizing only the Gaussians inside the current field-of-view sliding window is sufficient for a globally consistent map and accurate odometry over long trajectories, so Gaussians that leave the window never need to be re-optimized.

Editorial extensions

If this is right

  • At the reported update rates, the Gaussian map can be consumed online for occupancy-grid generation, obstacle avoidance, and novel-view rendering during a mission, not after it.
  • Because only field-of-view Gaussians occupy GPU memory while the global map lives in CPU RAM, environments larger than GPU memory can be mapped as long as system RAM or swap space is available.
  • Sequential IESKF updates let the visual rendering residual contribute a pose update with covariance that propagates to later IMU and LiDAR updates, making the odometry more robust to brief loss of one sensor.
  • Root voxel size, octree depth, image resolution, and sliding-window capacity form a direct trade-off between update frequency and map detail, allowing the same system to be configured for indoor and large-scale outdoor operation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not pursue is re-activating retired Gaussians when the robot revisits an area, which would add a loop-closure mechanism for the global map; without it, long-revisit global consistency remains an open question.
  • Because the visual measurement model only requires a rendered image and a photometric residual, the same sliding-window architecture could in principle wrap any differentiable radiance-field representation, not just 3D Gaussian Splatting.
  • The reported outdoor rendering quality (around 15 dB PSNR) is noticeably lower than indoor quality, suggesting large-scale outdoor scenes stress the fixed-level octree; a testable improvement is adaptive voxel sizing driven by range, texture, or structural complexity.
  • The integrated autonomous navigation demonstration indicates the Gaussian map can be projected to occupancy grids for planning; a natural next experiment is measuring how much map update rate, as opposed to pose accuracy, limits closed-loop control performance.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. GS-LIVO proposes a LiDAR-inertial-visual odometry and mapping system in which the map is represented by 3D Gaussians stored in a hash-indexed octree, with only the Gaussians in a sliding window optimized on the GPU. The paper claims real-time high-frequency map updates (over 10 Hz indoors and around 3 Hz outdoors) on an NVIDIA Jetson Orin NX, competitive odometry accuracy compared with FAST-LIVO, R3LIVE, and LVI-SAM, and reduced memory and optimization time compared with 3D-GS, SplaTAM, and MonoGS. Experiments are reported on subsets of the FAST-LIVO2 and MARS-LVIG datasets plus three indoor sequences, with PSNR for rendering quality and RMSE for odometry accuracy.

Significance. If validated, the contribution is genuinely useful: the memory-bounded split between a RAM-resident global octree and a VRAM-resident sliding window is a credible engineering solution for scalable Gaussian SLAM, and the IESKF visual update that renders the Gaussian map to form photometric residuals is a sensible extension of FAST-LIVO2. The paper also promises open-source release of algorithms, hardware designs, and CAD models, which would help the community. However, the current evidence is not sufficient to support the central claims: the outdoor RMSE numbers contradict each other, the PSNR evaluation is measured on the same frames used for optimization, and no experiment tests global consistency after revisiting a previously mapped area. The architecture is plausible, but the evaluation needs substantial strengthening.

major comments (4)
  1. [Section III.B.2, Table III] The outdoor accuracy claim is internally inconsistent and therefore not verifiable. The text states that for outdoor scenarios 'our method demonstrates superior accuracy with an RMSE of 0.042m', while Table III reports 0.58 m for HKisland03 and 0.63 m for HKairport01 for GS-LIVO, and Fig. 7 shows errors around 0.54 m RMSE. The reported numbers need to be reconciled, and the paper should state whether the 0.042 m value is a typo, a different sequence, or a different metric. No variance or repeated-run statistics are provided for any of the RMSE values.
  2. [Section II.C, Section II.D, Section III] The global-consistency claim of the SLAM system is not supported by any experiment. Section II.C retires Gaussians that leave the field of view by copying them into the global hash-indexed octree, and Section II.D's IESKF contains no loop-closure mechanism, so retired Gaussians are never re-optimized when the robot revisits a location. All odometry experiments in Section III are forward trajectories without a measured return to a previously mapped area. Please add a revisit or loop-closure experiment, or explicitly restrict the claims to local odometry with a local Gaussian map.
  3. [Section II.B.2, Section III.B.1, Table II] The PSNR comparison in Table II measures fit to the training views rather than novel-view synthesis. The Gaussians are optimized against the same images Ik used to compute PSNR through Eq. (9), and no held-out view synthesis metric is reported. The 'roaming perspective' images in Fig. 5 are qualitative only. Please report PSNR, SSIM, or LPIPS on held-out viewpoints or on frames not used in the optimization.
  4. [Section III.B.1 vs. Section III.B.2] The experimental configuration is described inconsistently. Section III.B.1 states that indoor scenes use a root voxel size of 0.03 m and outdoor scenes use 1.0 m, while Section III.B.2 states that the octree configuration is (0.06 m, 2 layers) for indoor and (0.5 m, 2 layers) for outdoor environments. These are materially different parameter choices and must be reconciled in a single parameter table so that the results in Tables II and III are reproducible.
minor comments (6)
  1. [Section III.B.2] The phrase 'as demonstrated in Tab. 7' refers to a nonexistent table; it should refer to Table III.
  2. [Table IV] The column header 'Dur./ms↑' uses an upward arrow, although lower duration is better; the arrow direction is inconsistent with Table II's 'Dur./s↓'.
  3. [Section III.A, Fig. 12] The text lists the outdoor sequences as HKisland03 and HKairport01, but Fig. 12 labels include 'HKisland01' and 'HKairport03'; these labels should be corrected.
  4. [Section II.B.1, Eq. (2)] The scale parameters sδ, sy, and sz in Eq. (2) are not fully defined; sδ is called a hyper-parameter, but the reader needs to know how sy and sz are obtained from the LiDAR downsampling or voxel level.
  5. [Section II.B.1, Eq. (3)] The sentence 'where ex is the normal vector on x-axis' is unclear; presumably ex is the unit vector of the x-axis, and the notation should be stated explicitly.
  6. [Table II] The entries marked ✗ for S3GS on HKU01, HKisland03, and HKairport01 are not explained; please state why the comparison is unavailable for those sequences.

Circularity Check

1 steps flagged · score 4.0 of 10

Odometry is externally validated against RTK/MoCap ground truth, but the rendering-quality PSNR is computed on the same frames used to optimize the Gaussians, making the mapping-quality sub-claim circular.

  1. fitted input called prediction [Section II.B.2, Eq. (9); Section III.B.1, Table II]
    "To refine the Gaussian map parameters, we consider the following optimization problem: θ∗k−1 = arg min θk−1 X ||Ik−1 − bIk−1(WTC; θk−1)|| (9) ... As shown in Table II, we first compare our approach with LiDAR-integrated Gaussian reconstruction methods such as S3Gaussian [60] and LetsGo [48]."

    Equation (9) optimizes the Gaussian parameters θk−1 by minimizing the photometric residual between the captured image Ik−1 and the image rendered from those same Gaussians. Table II reports PSNR values for GS-LIVO on the same dataset sequences after this optimization, and the paper describes no held-out or novel-view split in the evaluation protocol. Since PSNR is a monotone transform of the mean squared pixel error, the reported rendering quality measures fit to the training views, not generalization to new views. The mapping-quality claim therefore partially restates the optimized objective rather than providing independent evidence of photorealistic novel-view synthesis.

full rationale

The central odometry and real-time claims are not circular. Pose RMSE is checked against external RTK ground truth on MARS-LVIG sequences and motion-capture ground truth on the indoor sequences (Table III, Table IV, Fig. 7), so the localization results are independently falsifiable. The IESKF visual-update chain re-derives photometric Jacobians in the style of the authors' prior FAST-LIVO2 work, but reusing one's own framework is not circular when the output is validated against external ground truth. The sliding-window design is an efficiency mechanism, and the absence of a revisit or loop-closure experiment is an evidence gap about global consistency over long trajectories, not a circular derivation. The one genuine self-referential element is the rendering-quality evaluation: the PSNR numbers in Table II and Figs. 10-11 are computed from the same images that Eq. (9) uses as supervision, with no stated held-out split. Thus the 'high rendering quality' sub-claim partly reduces to the fitting loss itself. Because the odometry claim is externally validated and only the rendering sub-claim is self-measured, a moderate score of 4 is appropriate.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

All free parameters are engineering configurations, not physics constants fit to data; but several are tuned per dataset (voxel size, window size, resolution), which affects the comparative results. The main assumptions are borrowed correctness of 3DGS/MonoGS Jacobians and the sufficiency of local sliding-window optimization. No external benchmarks with held-out views are provided for the map quality claims.

free parameters (6)
  • root voxel size vs = 0.03 m (indoor), 0.06 m (indoor default), 0.5 m or 1.0 m (outdoor)
    Chosen per dataset (Section III.B, III.D); controls map density and the number of Gaussians, directly trading quality against memory and speed.
  • octree maximum subdivision level = 2 (all experiments)
    Set to 2 in all experiments; limits LoD granularity and affects detail and memory.
  • sliding window size = 100,000 Gaussians (desktop), 20,000 (embedded)
    Hand-picked to fit available GPU memory and target frame rate (Section III.B.2, III.D).
  • s_delta (slice thickness) = not specified
    Hyper-parameter in Eq. (2) that makes the 2D planar Gaussian a thin slice; value never given, so exact geometry is unspecified.
  • image resolution = 640x480 (desktop), 256x216 (embedded)
    Resolution chosen per platform; affects optimization cost and rendering quality.
  • optimization iterations per frame = 10
    Used in online optimization and in the 15,000-iteration offline comparison; affects runtime and accuracy.
assumptions (4)
  • domain assumption The Gaussian splatting rendering pipeline is differentiable, and the Jacobians for photometric loss with respect to pose and Gaussian parameters are as derived in MonoGS [51] and 3DGS [25].
    Invoked in Section II.D and Appendix A-B; if these Jacobians are wrong, the IESKF visual update and Gaussian optimization are invalid.
  • domain assumption The pinhole camera projection model with calibrated intrinsics and extrinsics, plus temporal synchronization via PPS, accurately models the sensor setup.
    Used throughout Sections II.A-II.D; calibration errors would bias both mapping and odometry.
  • domain assumption The IESKF framework from FAST-LIVO2 [8] provides a valid tightly coupled estimator; the paper modifies only the visual measurement model.
    Section II.D states the odometry is 'modified from FAST-LIVO2'; correctness of the base filter is taken as given.
  • domain assumption The sliding-window optimization converges to a useful local optimum of the photometric loss for the current field of view.
    Section II.C restricts optimization to a subset of Gaussians; there is no proof that this preserves global map consistency or avoids local minima.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GS-LIVO: Real-Time LiDAR, Inertial, and Visual Multi-sensor Fused Odometry with Gaussian Mapping." pith.science (2026). https://pith.science/paper/YWTFVGZX

@misc{pith2026250108672,
  author       = {Pith},
  title        = {Pith review of: GS-LIVO: Real-Time LiDAR, Inertial, and Visual Multi-sensor Fused Odometry with Gaussian Mapping},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YWTFVGZX}},
  note         = {Machine review of arXiv:2501.08672}
}
read the original abstract

In recent years, 3D Gaussian splatting (3D-GS) has emerged as a novel scene representation approach. However, existing vision-only 3D-GS methods often rely on hand-crafted heuristics for point-cloud densification and face challenges in handling occlusions and high GPU memory and computation consumption. LiDAR-Inertial-Visual (LIV) sensor configuration has demonstrated superior performance in localization and dense mapping by leveraging complementary sensing characteristics: rich texture information from cameras, precise geometric measurements from LiDAR, and high-frequency motion data from IMU. Inspired by this, we propose a novel real-time Gaussian-based simultaneous localization and mapping (SLAM) system. Our map system comprises a global Gaussian map and a sliding window of Gaussians, along with an IESKF-based odometry. The global Gaussian map consists of hash-indexed voxels organized in a recursive octree, effectively covering sparse spatial volumes while adapting to different levels of detail and scales. The Gaussian map is initialized through multi-sensor fusion and optimized with photometric gradients. Our system incrementally maintains a sliding window of Gaussians, significantly reducing GPU computation and memory consumption by only optimizing the map within the sliding window. Moreover, we implement a tightly coupled multi-sensor fusion odometry with an iterative error state Kalman filter (IESKF), leveraging real-time updating and rendering of the Gaussian map. Our system represents the first real-time Gaussian-based SLAM framework deployable on resource-constrained embedded systems, demonstrated on the NVIDIA Jetson Orin NX platform. The framework achieves real-time performance while maintaining robust multi-sensor fusion capabilities. All implementation algorithms, hardware designs, and CAD models will be publicly available.

Figures

Figures reproduced from arXiv: 2501.08672 by the authors.

Figure 1
Figure 1. Components of GS-LIVO for Large-Scale Scenar [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. System overview of GS-LIVO: a real-time LiDAR-Inertial-Visual odometry system with Gaussian Splatting-based [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. An overview of the procedures for incrementally updating the sliding window of Gaussians (detailed in Sec. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Comparison of map representation delicacy with patch [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Mapping results of three distinct real-world scenes (a)- (c). Top row: the rendering results from camera poses. Middle [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Performance comparison of different SLAM systems in terms of accuracy (RMSE) and computational efficiency [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Trajectory and error analysis for the sequence of [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Performance evaluation of GS-LIVO on the embedded platform: (a)- (d) system metrics including PSNR and processing [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: The relationship of the number of Gaussians with the [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 12
Figure 12. Figure 12: Time consumption analysis of the mapping process [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]
Figure 11
Figure 11. Figure 11: Performance analysis of the sliding window approach [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 47 canonical work pages

  1. [1]

    3D Gaussian Splatting Tutorial,

    G. Kopanas, B. Kerbl, A. Gu ´edon, and J. Luiten, “3D Gaussian Splatting Tutorial,” 2024, international Conference on 3D Vision Tutorial. [Online]. Available: https://3dgstutorial.github.io/

    work page 2024

  2. [2]

    Fast-lio: A fast, robust lidar-inertial odometry package by tightly-coupled iterated kalman filter,

    W. Xu and F. Zhang, “Fast-lio: A fast, robust lidar-inertial odometry package by tightly-coupled iterated kalman filter,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 3317–3324, 2021

    2021

  3. [3]

    Fast-lio2: Fast direct lidar- inertial odometry,

    W. Xu, Y . Cai, D. He, J. Lin, and F. Zhang, “Fast-lio2: Fast direct lidar- inertial odometry,” IEEE Transactions on Robotics , vol. 38, no. 4, pp. 2053–2073, 2022

    2022

  4. [4]

    R2live: A robust, real-time, lidar-inertial-visual tightly-coupled state estimator and mapping,

    J. Lin, C. Zheng, W. Xu, and F. Zhang, “R2live: A robust, real-time, lidar-inertial-visual tightly-coupled state estimator and mapping,” IEEE Robotics and Automation Letters , vol. 6, no. 4, pp. 7469–7476, 2021

    work page 2021

  5. [5]

    R3live: A robust, real-time, rgb-colored, lidar- inertial-visual tightly-coupled state estimation and mapping package,

    J. Lin and F. Zhang, “R3live: A robust, real-time, rgb-colored, lidar- inertial-visual tightly-coupled state estimation and mapping package,” in 2022 International Conference on Robotics and Automation (ICRA) . IEEE, 2022, pp. 10 672–10 678

    work page 2022

  6. [6]

    R$^3$LIVE++: A Robust, Real-time, Radiance reconstruction package with a tightly-coupled LiDAR-Inertial-Visual state Estimator

    ——, “R3live++: A robust, real-time, radiance reconstruction pack- age with a tightly-coupled lidar-inertial-visual state estimator,” arXiv preprint arXiv:2209.03666, 2022

    work page Pith review arXiv 2022

  7. [7]

    Fast-livo: Fast and tightly-coupled sparse-direct lidar-inertial-visual odometry,

    C. Zheng, Q. Zhu, W. Xu, X. Liu, Q. Guo, and F. Zhang, “Fast-livo: Fast and tightly-coupled sparse-direct lidar-inertial-visual odometry,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, pp. 4003–4009

    work page 2022

  8. [8]

    Fast-livo2: Fast, direct lidar-inertial-visual odometry,

    C. Zheng, W. Xu, Z. Zou, T. Hua, C. Yuan, D. He, B. Zhou, Z. Liu, et al. , “Fast-livo2: Fast, direct lidar-inertial-visual odometry,” IEEE Transactions on Robotics , 2024

    work page 2024

Show all 67 references
  1. [9]

    Mars-lvig dataset: A multi-sensor aerial robots slam dataset for lidar-visual-inertial-gnss fusion,

    H. Li, Y . Zou, N. Chen, J. Lin, X. Liu, W. Xu, C. Zheng, R. Li, D. He, F. Kong, et al. , “Mars-lvig dataset: A multi-sensor aerial robots slam dataset for lidar-visual-inertial-gnss fusion,” The International Journal of Robotics Research , p. 02783649241227968, 2024

    2024

  2. [10]

    SVO: Semidirect visual odometry for monocular and multicamera systems,

    C. Forster, Z. Zhang, M. Gassner, M. Werlberger, and D. Scaramuzza, “SVO: Semidirect visual odometry for monocular and multicamera systems,” IEEE Trans. Robot. , vol. 33, no. 2, pp. 249–265, 2017

    2017

  3. [11]

    Immesh: An immediate lidar localization and meshing framework,

    J. Lin, C. Yuan, Y . Cai, H. Li, Y . Ren, Y . Zou, X. Hong, and F. Zhang, “Immesh: An immediate lidar localization and meshing framework,” IEEE Transactions on Robotics , 2023

    2023

  4. [12]

    Cad-mesher: A convenient, accurate, dense mesh-based mapping module in slam for dynamic environments,

    Y . Jia, F. Cao, T. Wang, Y . Tang, S. Shao, and L. Liu, “Cad-mesher: A convenient, accurate, dense mesh-based mapping module in slam for dynamic environments,” arXiv preprint arXiv:2408.05981 , 2024

    2024 arXiv

  5. [13]

    Mesh-loam: Real-time mesh-based lidar odometry and mapping,

    Y . Zhu, X. Zheng, and J. Zhu, “Mesh-loam: Real-time mesh-based lidar odometry and mapping,” IEEE Transactions on Intelligent Vehicles, 2024

    2024

  6. [14]

    Slamesh: Real-time lidar simultaneous localization and meshing,

    J. Ruan, B. Li, Y . Wang, and Y . Sun, “Slamesh: Real-time lidar simultaneous localization and meshing,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2023, pp. 3546–3552

    2023

  7. [15]

    Simplified unstructured-mesh based uav path planning method using octree overlap detection,

    W. Wang, H. Qian, and S. Feng, “Simplified unstructured-mesh based uav path planning method using octree overlap detection,” IEEE Trans- actions on Intelligent Vehicles , 2024

    2024

  8. [16]

    Pmds-slam: Probability mesh enhanced semantic slam in dynamic environments,

    C. Wang, Y . Zhang, and X. Li, “Pmds-slam: Probability mesh enhanced semantic slam in dynamic environments,” in 2020 5th International Conference on Control, Robotics and Cybernetics (CRC) . IEEE, 2020, pp. 40–44

    2020

  9. [17]

    Sp-slam: Surfel-point simultaneous localization and mapping,

    H. M. Cho, H. Jo, and E. Kim, “Sp-slam: Surfel-point simultaneous localization and mapping,” IEEE/ASME Transactions on Mechatronics , vol. 27, no. 5, pp. 2568–2579, 2021

    2021

  10. [18]

    Real-time multi-adaptive-resolution-surfel 6d lidar odometry using continuous-time trajectory optimization,

    J. Quenzel and S. Behnke, “Real-time multi-adaptive-resolution-surfel 6d lidar odometry using continuous-time trajectory optimization,” in 2021 IEEE/RSJ international conference on intelligent robots and sys- tems (IROS). IEEE, 2021, pp. 5499–5506

    2021

  11. [19]

    Slict: Multi- input multi-scale surfel-based lidar-inertial continuous-time odometry and mapping,

    T.-M. Nguyen, D. Duberg, P. Jensfelt, S. Yuan, and L. Xie, “Slict: Multi- input multi-scale surfel-based lidar-inertial continuous-time odometry and mapping,” IEEE Robotics and Automation Letters , vol. 8, no. 4, pp. 2102–2109, 2023

    2023

  12. [20]

    Real-time scalable dense surfel mapping,

    K. Wang, F. Gao, and S. Shen, “Real-time scalable dense surfel mapping,” in 2019 International conference on robotics and automation (ICRA). IEEE, 2019, pp. 6919–6925

    2019

  13. [21]

    Surfelnerf: Neural surfel radiance fields for online photorealistic reconstruction of indoor scenes,

    Y . Gao, Y .-P. Cao, and Y . Shan, “Surfelnerf: Neural surfel radiance fields for online photorealistic reconstruction of indoor scenes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 108–118

    2023

  14. [22]

    Autonomous aerial robot using dual-fisheye cameras,

    W. Gao, K. Wang, W. Ding, F. Gao, T. Qin, and S. Shen, “Autonomous aerial robot using dual-fisheye cameras,” Journal of Field Robotics , vol. 37, no. 4, pp. 497–514, 2020

    2020

  15. [23]

    Vins-mono: A robust and versatile monocular visual-inertial state estimator,

    T. Qin, P. Li, and S. Shen, “Vins-mono: A robust and versatile monocular visual-inertial state estimator,” IEEE Transactions on Robotics , vol. 34, no. 4, pp. 1004–1020, 2018

    2018

  16. [24]

    Mildenhall, P

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view Optimization Time vs. Number of Gaussians (10 Iterations) 0 1 2 3 4 5 Number of Gaussians in Sliding Window105 0.01 0.02 0.03 0.04...

    2021

  17. [25]

    3d gaussian splatting for real-time radiance field rendering,

    B. Kerbl, G. Kopanas, T. Leimk ¨uhler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering,” ACM Transactions on Graphics (ToG), vol. 42, no. 4, pp. 1–14, 2023

    2023

  18. [26]

    A hierarchical 3d gaussian representation for real-time rendering of very large datasets,

    B. Kerbl, A. Meuleman, G. Kopanas, M. Wimmer, A. Lanvin, and G. Drettakis, “A hierarchical 3d gaussian representation for real-time rendering of very large datasets,” ACM Transactions on Graphics , vol. 43, no. 4, July 2024. [Online]. Available: https://repo-sam.inria.fr/fungr...

    2024

  19. [27]

    iMAP: Implicit mapping and positioning in real-time,

    E. Sucar, S. Liu, J. Ortiz, and A. Davison, “iMAP: Implicit mapping and positioning in real-time,” in Proceedings of the International Conference on Computer Vision (ICCV) , 2021

    2021

  20. [28]

    Nice-slam: Neural implicit scalable encoding for slam,

    Z. Zhu, S. Peng, V . Larsson, W. Xu, H. Bao, Z. Cui, M. R. Oswald, and M. Pollefeys, “Nice-slam: Neural implicit scalable encoding for slam,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 12 786–12 796

    2022

  21. [29]

    Co-slam: Joint coordinate and sparse parametric encodings for neural real-time slam,

    H. Wang, J. Wang, and L. Agapito, “Co-slam: Joint coordinate and sparse parametric encodings for neural real-time slam,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 13 293–13 302

    2023

  22. [30]

    Eslam: Efficient dense slam system based on hybrid representation of signed distance fields,

    M. M. Johari, C. Carta, and F. Fleuret, “Eslam: Efficient dense slam system based on hybrid representation of signed distance fields,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 17 408–17 419

    2023

  23. [31]

    Uncle-slam: Uncer- tainty learning for dense neural slam,

    E. Sandstr ¨om, K. Ta, L. V . Gool, and M. R. Oswald, “Uncle-slam: Uncer- tainty learning for dense neural slam,” arXiv preprint arXiv:2306.11048, 2023

    2023 arXiv

  24. [32]

    Chung, Y .-C

    C.-M. Chung, Y .-C. Tseng, Y .-C. Hsu, X.-Q. Shi, Y .-H. Hua, J.-F. Yeh, W.-C. Chen, Y .-T. Chen, and W. H. Hsu, “Orbeez-slam: A real-time 0 200 400 600 Frame Index 0 5 10 15 20 25 30 35PSNR (dB) (a) PSNR Comparison 24.09 23.94 w/ SW w/o SW 0 200 400 600 Frame Index 0 20 40 60...

    2000 arXiv

  25. [33]

    Splatam: Splat track & map 3d gaussians for dense rgb-d slam,

    N. Keetha, J. Karhade, K. M. Jatavallabhula, G. Yang, S. Scherer, D. Ramanan, and J. Luiten, “Splatam: Splat track & map 3d gaussians for dense rgb-d slam,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 21 357–21 366

    2024

  26. [34]

    Gs-slam: Dense visual slam with 3d gaussian splatting,

    C. Yan, D. Qu, D. Xu, B. Zhao, Z. Wang, D. Wang, and X. Li, “Gs-slam: Dense visual slam with 3d gaussian splatting,” in CVPR, 2024

    2024

  27. [35]

    Mm3dgs slam: Multi-modal 3d gaussian splatting for slam using vision, depth, and inertial measurements,

    L. C. Sun, N. P. Bhatt, J. C. Liu, Z. Fan, Z. Wang, T. E. Humphreys, and U. Topcu, “Mm3dgs slam: Multi-modal 3d gaussian splatting for slam using vision, depth, and inertial measurements,” 2024

    2024

  28. [36]

    Liv-gs: Lidar-vision integration for 3d gaussian splatting slam in outdoor environments,

    R. Xiao, W. Liu, Y . Chen, and L. Hu, “Liv-gs: Lidar-vision integration for 3d gaussian splatting slam in outdoor environments,” IEEE Robotics and Automation Letters , 2024

    2024

  29. [37]

    Liv-gaussmap: Lidar- inertial-visual fusion for real-time 3d radiance field map rendering,

    S. Hong, J. He, X. Zheng, C. Zheng, and S. Shen, “Liv-gaussmap: Lidar- inertial-visual fusion for real-time 3d radiance field map rendering,” arXiv preprint arXiv:2401.14857 , 2024

    2024 arXiv

  30. [38]

    Instant neural graphics primitives with a multiresolution hash encoding,

    T. M ¨uller, A. Evans, C. Schied, and A. Keller, “Instant neural graphics primitives with a multiresolution hash encoding,” ACM Trans. Graph. , vol. 41, no. 4, pp. 102:1–102:15, July 2022. [Online]. Available: https://doi.org/10.1145/3528223.3530127

    2022

  31. [39]

    ORB-SLAM3: An accurate open-source library for visual, visual-inertial and multi-map SLAM,

    C. Campos, R. Elvira, J. J. Gomez, J. M. M. Montiel, and J. D. Tardos, “ORB-SLAM3: An accurate open-source library for visual, visual-inertial and multi-map SLAM,” IEEE Transactions on Robotics , vol. 37, no. 6, pp. 1874–1890, 2021. 0 500 1000 1500 2000 2500 Frame Index 0 5 10...

    2021

  32. [40]

    H2-mapping: Real-time dense mapping using hierarchical hybrid rep- resentation,

    C. Jiang, H. Zhang, P. Liu, Z. Yu, H. Cheng, B. Zhou, and S. Shen, “H2-mapping: Real-time dense mapping using hierarchical hybrid rep- resentation,” IEEE Robotics and Automation Letters , vol. 8, no. 10, pp. 6787–6794, 2023

    2023

  33. [41]

    H3-mapping: Quasi- heterogeneous feature grids for real-time dense mapping using hierar- chical hybrid representation,

    C. Jiang, Y . Luo, B. Zhou, and S. Shen, “H3-mapping: Quasi- heterogeneous feature grids for real-time dense mapping using hierar- chical hybrid representation,” arXiv preprint arXiv:2403.10821 , 2024

    2024 arXiv

  34. [42]

    Swift- mapping: Online neural implicit dense mapping in urban scenes,

    K. Wu, K. Zhang, M. Gao, J. Zhao, Z. Gan, and W. Ding, “Swift- mapping: Online neural implicit dense mapping in urban scenes,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 6, 2024, pp. 6048–6056

    2024

  35. [43]

    Motiongs: Compact gaussian splatting slam by motion filter,

    X. Guo, P. Han, W. Zhang, and H. Chen, “Motiongs: Compact gaussian splatting slam by motion filter,” arXiv preprint arXiv:2405.11129, 2024

    2024 arXiv

  36. [44]

    Rtg- slam: Real-time 3d reconstruction at scale using gaussian splatting,

    Z. Peng, T. Shao, L. Yong, J. Zhou, Y . Yang, J. Wang, and K. Zhou, “Rtg- slam: Real-time 3d reconstruction at scale using gaussian splatting,” 2024

    2024

  37. [45]

    Octree-gs: To- wards consistent real-time rendering with lod-structured 3d gaussians,

    K. Ren, L. Jiang, T. Lu, M. Yu, L. Xu, Z. Ni, and B. Dai, “Octree-gs: To- wards consistent real-time rendering with lod-structured 3d gaussians,” arXiv preprint arXiv:2403.17898 , 2024

    2024 arXiv

  38. [46]

    Real- time view synthesis for large scenes with millions of square meters,

    Q. Shuai, H. Guo, Z. Xu, H. Lin, S. Peng, H. Bao, and X. Zhou, “Real- time view synthesis for large scenes with millions of square meters,” 2024

    2024

  39. [47]

    Gaussian-lic: Photo-realistic lidar-inertial-camera slam with 3d gaussian splatting,

    X. Lang, L. Li, H. Zhang, F. Xiong, M. Xu, Y . Liu, X. Zuo, and J. Lv, “Gaussian-lic: Photo-realistic lidar-inertial-camera slam with 3d gaussian splatting,” arXiv preprint arXiv:2404.06926 , 2024. UAV playground01UAV playground02 Landmark01 CBD Building 01 HKU01 HKisland01HKa...

    2024 arXiv

  40. [48]

    Letsgo: Large-scale garage modeling and rendering via lidar-assisted gaussian primitives,

    J. Cui, J. Cao, F. Zhao, Z. He, Y . Chen, Y . Zhong, L. Xu, Y . Shi, Y . Zhang, and J. Yu, “Letsgo: Large-scale garage modeling and rendering via lidar-assisted gaussian primitives,” ACM Trans. Graph. , vol. 43, no. 6, Nov. 2024. [Online]. Available: https://doi.org/10.1145/3687762

    2024 doi

  41. [49]

    Li- gs: Gaussian splatting with lidar incorporated for accurate large-scale reconstruction,

    C. Jiang, R. Gao, K. Shao, Y . Wang, R. Xiong, and Y . Zhang, “Li- gs: Gaussian splatting with lidar incorporated for accurate large-scale reconstruction,” arXiv preprint arXiv:2409.12899 , 2024

    2024 arXiv

  42. [50]

    Photo-slam: Real-time simultaneous localization and photorealistic mapping for monocular, stereo, and rgb-d cameras,

    H. Huang, L. Li, C. Hui, and S.-K. Yeung, “Photo-slam: Real-time simultaneous localization and photorealistic mapping for monocular, stereo, and rgb-d cameras,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024

    2024

  43. [51]

    Gaussian Splatting SLAM,

    H. Matsuki, R. Murai, P. H. J. Kelly, and A. J. Davison, “Gaussian Splatting SLAM,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024

    2024

  44. [52]

    Lic-fusion: Lidar- inertial-camera odometry,

    X. Zuo, P. Geneva, W. Lee, Y . Liu, and G. Huang, “Lic-fusion: Lidar- inertial-camera odometry,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2019, pp. 5848–5854

    2019

  45. [53]

    Camvox: A low-cost and accurate lidar-assisted visual slam system,

    Y . Zhu, C. Zheng, C. Yuan, X. Huang, and X. Hong, “Camvox: A low-cost and accurate lidar-assisted visual slam system,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 5049–5055

    2021

  46. [54]

    Lvi-sam: Tightly-coupled lidar-visual-inertial odometry via smoothing and mapping,

    T. Shan, B. Englot, C. Ratti, and D. Rus, “Lvi-sam: Tightly-coupled lidar-visual-inertial odometry via smoothing and mapping,” in 2021 IEEE international conference on robotics and automation (ICRA) . IEEE, 2021, pp. 5692–5698

    2021

  47. [55]

    Rollvox: Real-time and high- quality lidar colorization with rolling shutter camera,

    S. Hong, C. Zheng, H. Yin, and S. Shen, “Rollvox: Real-time and high- quality lidar colorization with rolling shutter camera,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2023, pp. 7195–7201

    2023

  48. [56]

    Efficient and probabilistic adaptive voxel mapping for accurate online lidar odometry,

    C. Yuan, W. Xu, X. Liu, X. Hong, and F. Zhang, “Efficient and probabilistic adaptive voxel mapping for accurate online lidar odometry,” IEEE Robotics and Automation Letters , vol. 7, no. 3, pp. 8518–8525, 2022

    2022

  49. [57]

    Surface splatting,

    M. Zwicker, H. Pfister, J. Van Baar, and M. Gross, “Surface splatting,” in Proceedings of the 28th annual conference on Computer graphics and interactive techniques , 2001, pp. 371–378

    2001

  50. [58]

    Mfcalib: Single-shot and automatic extrinsic calibration for lidar and camera in targetless environments based on multi-feature edge,

    T. Ye, W. Xu, C. Zheng, and Y . Cui, “Mfcalib: Single-shot and automatic extrinsic calibration for lidar and camera in targetless environments based on multi-feature edge,” arXiv preprint arXiv:2409.00992 , 2024

    2024 arXiv

  51. [59]

    Evaluation of combined time-offset estimation and hand- eye calibration on robotic datasets,

    F. Furrer, M. Fehr, T. Novkovic, H. Sommer, I. Gilitschenski, and R. Siegwart, “Evaluation of combined time-offset estimation and hand- eye calibration on robotic datasets,” in Field and Service Robotics: Results of the 11th International Conference . Springer, 2018, pp. 145– 159

    2018

  52. [60]

    S3gaussian: Self-supervised street gaussians for autonomous driving,

    N. Huang, X. Wei, W. Zheng, P. An, M. Lu, W. Zhan, M. Tomizuka, K. Keutzer, and S. Zhang, “S3gaussian: Self-supervised street gaussians for autonomous driving,” arXiv preprint arXiv:2405.20323 , 2024. APPENDIX This appendix presents the complete derivation chain from photometr...

    2024 arXiv

  53. [61]

    Mean Value Jacobians: For the mean value component: ∂qi ∂W RI = ∂qi ∂Cpi ∂Cpi ∂CRW ∂CRW ∂W RI (18) and ∂qi ∂W tI = ∂qi ∂Cpi ∂Cpi ∂CtW ∂CtW ∂W tI + ∂qi ∂Cpi ∂Cpi ∂CRW ∂CRW ∂W tI (19)

  54. [62]

    Camera-IMU Transform Jacobians To complete the above chains, we need to derive ∂CRW ∂W RI , ∂CtW ∂W tI , and ∂CRW ∂W tI using the perturbation method

    Covariance Jacobians: For the 2D covariance terms: ∂Σ2D ∂W RI = ∂Σ2D ∂Jπ ∂Jπ ∂Cpi ∂Cpi ∂CRW ∂CRW ∂W RI + ∂Σ2D ∂CRW ∂CRW ∂W RI (20) and ∂Σ2D ∂W tI = ∂Σ2D ∂Jπ ∂Jπ ∂Cpi ∂Cpi ∂CtW ∂CtW ∂W tI + ∂Σ2D ∂Jπ ∂Jπ ∂Cpi ∂Cpi ∂CRW ∂CRW ∂W tI (21) B. Camera-IMU Transform Jacobians To complet...

  55. [63]

    This equation can be decomposed into rotation and translation parts: (Exp(δφ∧)CRW ) −1CRI = W RI · Exp(δR∧) CRW T (Exp(−δφ∧)(CtI − δt)CtW ) = (W tI + δρ) (23)

    Perturbation Model: The fundamental relationship be- tween camera and IMU poses under perturbation is: (T(δφ, δt) · CTW )−1 · CTI = W TI ⊞ T(δR, δρ) (22) where (δφ, δt) represents perturbations on camera pose CTW and ξ = (δR, δρ) represents perturbations on IMU pose W TI. This...

  56. [64]

    Rotation Component: Starting from the rotation equation in (23): Taking inverse and substituting: W RCExp(−δφ∧)CRI = W RI Exp(δR∧) (24) Using first-order approximation Exp(ξ∧) ≈ I + ξ∧: W RI − W RCδφ∧CRI =W RI +W RI δR∧ (25) After cancellation: −I RCδφ∧CRI = δR∧ (26) (−I RCδφ)...

  57. [65]

    IESKF Update Framework

    Translation Component: Starting from the translation equation in (23): Expanding first-order terms: W RC(I − δφ∧)CtI − W RC(I − δφ∧)(CtW + δt) =W tI + δρ (30) After algebraic simplification: W RC C t∧ I δφ − W RCδt = δρ (31) Therefore: ∂CtW ∂W tI = −CRW (32) ∂CRW ∂W tI = −Ct∧ ...

  58. [66]

    Prior and Posterior State: The state consists of: • Prior state: ¯W TI with covariance Σ ¯W TI • Posterior state: ˇW TI with covariance Σ ˇW TI The Jacobians derived in Section B relate to Ji in the mea- surement model: M( ˆW TI , ui) + Jiξ (34)

  59. [67]

    Update Process: The IESKF update optimizes: arg min δ ˆW TI X ui M( ˆW TI , ui) + JiT (ξ) 2 Σui + ˆW TI ⊟ ¯W TI + HT (ξ) 2 Σ ¯W TI (35) The solution follows: H = [JT 1 , · · ·, JT m]T (36) R = diag(Σu1, · · ·, Σum) (37) P = H−1Σ ¯W TI H−T (38) K = (HT R−1H + P−1)−1HT R−1 (39) ...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.