Pith. sign in

REVIEW 3 major objections 5 minor 27 references

Towards Explaining Uncertainty Estimates in Point Cloud Registration

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Kernel SHAP can attribute probabilistic ICP's pose uncertainty to sensor noise, initial pose error, or partial overlap, and in seven of eight real-world sequences sensor noise is the dominant source.

desk verdict A first application of kernel SHAP to ICP uncertainty that is plausible as a proof of concept but carries an unexamined zero-baseline assumption that may shift every attribution. read the letter →

arxiv 2412.20612 v1 pith:ELTBDLEZ submitted 2024-12-29 cs.RO cs.AI

Ziyuan Qin , Jongseok Lee , Rudolph Triebel This is my paper · ORCID
classification cs.ROcs.AI
keywords pointcloudregistrationIterativeClosest(ICP)uncertaintyestimationexplainableAISHAPkernelsensornoisepose
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that kernel SHAP can explain why a probabilistic ICP point cloud registration algorithm reports high pose uncertainty, by attributing that uncertainty to three named sources: sensor noise, initial pose uncertainty, and partial overlap. The authors model each source as a perturbation of the ICP input, run ICP under the eight possible perturbation coalitions, and fit a weighted linear model whose coefficients are SHAP values. Across eight real-world sequences, the attributions are mostly nonnegative, and sensor noise emerges as the dominant contributor in seven of eight cases. The paper concludes that the explanations are reasonable and human interpretable, a step toward robots that know when and why they failed.

What carries the argument

The central object is the Shapley kernel applied to three binary feature indicators that encode whether sensor noise, initial pose uncertainty, or partial overlap is present. Each coalition is mapped to a concrete perturbation magnitude via the reference value $\{0,1,0\}$, ICP is run to produce 100 pose samples, and the pose uncertainty is quantified as the KL divergence between the perturbed and pseudo-true 100-sample distributions. The weighted linear regression with the Shapley kernel produces the unique additive attribution satisfying local accuracy, missingness, and consistency, and its closed-form solution yields the SHAP values $\phi_{\text{sn}}$, $\phi_{\text{ip}}$, $\phi_{\text{po}}$ for the three uncertainty sources.

What would settle it

Compute the KL divergence between two independent 100-sample sets of ICP pose estimates drawn from the same unperturbed inputs, with identical sensor noise, initial pose scale, and overlap ratio. If the resulting values are not negligible compared with the SHAP attributions, the zero-baseline assumption fails and the reported SHAP values would shift.

Watch

Extended reading notes

Core claim

The paper's central claim is that uncertainty estimates in ICP can be explained by additive feature attribution over the perturbation magnitudes of three uncertainty sources, using kernel SHAP. Given an instance with sensor noise $\sigma$, initial pose scale $s$, and overlap removal $\lambda$, the method samples 100 ICP pose estimates for each of the $2^3$ coalitions, computes the KL divergence between the perturbed pose distribution and a pseudo-true unperturbed distribution as the uncertainty output, and solves a weighted linear regression with the Shapley kernel to obtain attributions $\phi_{\text{sn}}$, $\phi_{\text{ip}}$, $\phi_{\text{po}}$. The experiments show that these attributions are mostly positive, increase with feature values, and exhibit interpretable interaction effects, such as high initial pose uncertainty overshadowing sensor noise. The authors state that the method reasonably explains the uncertainty sources, while cautioning that the explanations assess correlation, not causality.

Load-bearing premise

The entire attribution is anchored to the assumption that the unperturbed input has exactly zero pose uncertainty, so the SHAP bias term $\phi_0$ is omitted; with finite 100-sample distributions, a second independent sample of the same unperturbed input will have slightly different covariance and hence nonzero KL divergence.

Editorial extensions

If this is right

  • A robot using any uncertainty-aware ICP variant can, at run time, receive a ranked list of the uncertainty sources it should mitigate, with sensor noise at the top in most tested environments.
  • Because kernel SHAP is model-agnostic, the same pipeline can be attached to other ICP algorithms and to additional uncertainty sources such as under-constrained geometry without changing the explanation formalism.
  • In teleoperation, an operator told that sensor noise contributed most to this alignment uncertainty can choose a corrective action such as closing range, re-sensing, or denoising, rather than restarting blind.
  • The interaction effects shown in the dependence plots suggest that fixing the dominant source may be most effective: when initial pose uncertainty is already high, reducing sensor noise yields less benefit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same attribution pipeline could be applied to learning-based pose estimators with predictive variance, as long as a sampling-based uncertainty output is available; the perturbation-to-coalition mapping would carry over directly.
  • A natural testable extension would be a closed-loop experiment where the robot removes the top-attributed source and measures whether the pose uncertainty drops; if it does not, the attribution is not causally faithful.
  • The dominance of sensor noise in the median SHAP values may partly reflect the choice of perturbation ranges, so the ranking should be read as relative to the specified perturbation regime, not as an absolute property of the scenes.
  • Since the paper leaves causality unresolved, SHAP values here are diagnostic summaries; converting them into causal claims would require interventional data or a structured causal model of the registration pipeline.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper addresses the interpretability of uncertainty estimates produced by a probabilistic ICP pipeline. The authors identify three sources of pose uncertainty (sensor noise, initial pose uncertainty, partial overlap) and propose to attribute an estimated pose-uncertainty scalar to these sources via kernel SHAP. The uncertainty scalar is computed as the KL divergence between two 100-sample distributions of ICP pose estimates obtained under a 'pseudo-true' and a perturbed setting. The experiments on the Challenging datasets include SHAP summary, waterfall, and dependence plots for one sequence pair, and a median SHAP table across eight sequences. The authors conclude that the explanation method can reasonably identify which uncertainty source dominates.

Significance. If the result were fully validated, this would be a useful step toward human-interpretable failure analysis for point cloud registration, as operators could see whether sensor noise, initial pose error, or partial overlap drives the reported pose uncertainty. The paper applies kernel SHAP, a principled additive attribution method, in a domain where it has not been used before, and the experimental setup covers a realistic multi-sequence dataset. The authors also make the standard choice to explain only three of the five known uncertainty sources and state this limitation explicitly. However, the central validation is qualitative, and one load-bearing technical assumption (the zero-bias baseline) is not met in the implementation; the significance is thus conditional on correcting that assumption and providing a more quantitative evaluation of explanation quality.

major comments (3)
  1. [Section 5, Algorithm 1, Section 6] The claim that the bias phi0 can be omitted because the expected value for the unperturbed case is 0 is contradicted by the experimental setup. Algorithm 1 sets the reference values to r = {0, 1, 0}, so the reference input has initial-pose scale s = 1, not zero; moreover, the pseudo-true distribution in Section 6 is itself sampled with s = 1. Since the pose uncertainty is the KL divergence between two 100-sample distributions, f(r) is generally positive even for unperturbed inputs, because two finite samples from the same input distribution yield different sample covariances. The intercept-free weighted linear regression then forces this nonzero baseline into the three feature coefficients, biasing every SHAP value. This issue directly affects the attribution ordering reported in Table 1 and must be resolved before the central claim can be assessed.
  2. [Section 6, Pose Uncertainty Estimation] The definition of the model output f is under-specified. The text says the uncertainty is obtained by 'comparing the covariance matrices of these two normal distributions via KL divergence', but it does not provide the KL formula, the coordinate parameterization of the pose distributions (e.g., R^6 via se(3) or a manifold-valued Gaussian), nor the procedure for converting the 100 sampled poses into the two normal distributions. It also does not state how singular or near-singular covariance matrices are handled. Because kernel SHAP explains this scalar f, the paper must give a complete, reproducible mathematical definition of f.
  3. [Sections 6.1 and 6.2] The evidence for the central claim ('can reasonably explain the uncertainty sources') is qualitative. The summary, waterfall, and dependence plots are interpreted subjectively, negative SHAP values in Figure 2 are mentioned but not analyzed, and Table 1 only reports medians. No quantitative criterion (e.g., faithfulness of the additive model, agreement with controlled single-source perturbation responses, or comparison against a baseline attribution method such as LIME or simple sensitivity analysis) is provided. Without such a criterion, the conclusion that the method 'can reasonably explain' the uncertainty sources is not established beyond a descriptive level.
minor comments (5)
  1. [Algorithm 1] Algorithm 1 contains the placeholder 'see details in Section ??'; this cross-reference should be completed.
  2. [Section 3.3] 'More details are discuss in our experiments later' contains a grammatical error ('discuss' should be 'discussed').
  3. [Section 3.2] 'the uncertainty of ICP uncertainty comes mainly' is a typo; the duplicated word should be removed.
  4. [Section 6.1] The selection of point clouds 6 and 7 from the Apartment sequence is not motivated; a brief explanation of how these pairs are chosen would improve reproducibility.
  5. [Section 5 and Figure 1] The mapping function h_x is only described qualitatively; a precise definition of h_x for each feature (e.g., how z' = 0 maps to the reference value and z' = 1 maps to the perturbed value) would reduce ambiguity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: kernel SHAP is applied to an independently defined KL-uncertainty function; the omitted-bias assumption is a statistical modeling choice, not a self-referential derivation.

full rationale

The paper's derivation chain is self-contained in the sense required here. The target quantity to be explained is an uncertainty estimate defined as the KL divergence between a pseudo-true ICP pose distribution (sampled with baseline perturbations) and a perturbed pose distribution (Section 6, 'Pose Uncertainty Estimation'). Kernel SHAP is then used in its standard role: it evaluates this fixed function at all 2^3 coalitions of the three perturbation features and fits the additive model of Equation (5) by weighted linear regression (Algorithm 1). There is no fitted parameter that is later renamed as a prediction; the SHAP values are computed from the uncertainty function, not used to define it. The statement that the bias phi0 is omitted 'because the expected value for the unperturbed case is 0' is consistent with the paper's definition of the pseudo-true distribution as the unperturbed input, so the divergence is zero in expectation; finite-sample nonzero KL is a possible estimation bias, not a circular reduction. Self-citations to the authors' prior work appear only in the future-work paragraph (References [23]–[27]) and are not load-bearing for the central claim. The unresolved 'Section ??' pointer and the acknowledged lack of causal claims are manuscript defects and limitations, not circularity. No quoted equation or algorithmic step makes the conclusion equivalent to an input by construction.

Assumptions & free parameters 5 free parameters · 7 assumptions · 0 invented entities

The central claim rests on standard SHAP theory plus several domain-specific modeling choices: the perturbation schemes for the three uncertainty sources and the KL-based uncertainty target. None of these are fitted to the explanation results, but all are hand-chosen and affect the SHAP values. The most fragile assumption is that the unperturbed reference has exactly zero uncertainty, which is not true under finite sampling.

free parameters (5)
  • sensor noise sigma range = 0 to 0.1 m, step 0.01 m
    Hand-chosen perturbation range for Gaussian noise added to input point clouds; the SHAP attributions depend on the tested range.
  • initial pose scale s range = 1 to 2, step 0.1
    Hand-chosen range for the scale of the initial pose covariance; affects which feature dominates in the experiments.
  • partial overlap reduction lambda range = 0 to 0.1, step 0.01
    Hand-chosen range for the reduction in overlap ratio; affects the SHAP values.
  • overlap distance threshold d = 0.2 m
    Hand-chosen threshold for defining valid nearest-neighbor correspondences when computing overlap ratio; changes the definition of partial overlap.
  • number of ICP samples per distribution = 100
    Chosen by hand; determines the noise in the KL divergence estimate and hence in the SHAP values.
assumptions (7)
  • standard math Kernel SHAP computes the unique additive feature attribution satisfying local accuracy, missingness, and consistency.
    Invoked in Section 2 and used in Algorithm 1; standard background result from Lundberg and Lee [10].
  • domain assumption ICP pose uncertainty can be quantified by sampling pose estimates and comparing covariance matrices via KL divergence.
    Section 3.3 and Section 6 'Pose Uncertainty Estimation' assume the sampled ICP pose estimates form a normal distribution on SE(3) whose covariance can be compared via KL divergence.
  • domain assumption Sensor noise is represented as zero-mean Gaussian noise.
    Section 4.0.1: 'Since the sensor white noise is more straightforward to model, it is represented as a zero-mean Gaussian noise without loss of generality.' This ignores sensor bias noise.
  • domain assumption Initial pose perturbations follow a concentrated Gaussian distribution on SE(3).
    Section 4.0.2 uses T_init = T_gt exp(xi^) with xi ~ N(0, Sigma) (cf. [21]); the covariance Sigma is never specified or measured.
  • domain assumption Partial overlap can be perturbed by removing points from the overlap region with a fixed distance threshold.
    Section 4.0.3 defines overlap via kNN with d = 0.2 m and perturbs the point cloud by deleting points to reach a target overlap ratio.
  • ad hoc to paper Only three of five uncertainty sources need to be explained.
    Section 3.2 footnote: under-constrained situations and intrinsic ICP randomness are not explained 'for the sake of simplicity'; the paper assumes the remaining three sources are sufficient.
  • ad hoc to paper The SHAP bias can be omitted because the unperturbed case has zero uncertainty.
    Section 5: 'the bias phi0 is omitted because the expected value for the unperturbed case is 0'; this is an assumption about the reference point of the uncertainty model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Explaining Uncertainty Estimates in Point Cloud Registration." pith.science (2026). https://pith.science/paper/ELTBDLEZ

@misc{pith2026241220612,
  author       = {Pith},
  title        = {Pith review of: Towards Explaining Uncertainty Estimates in Point Cloud Registration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ELTBDLEZ}},
  note         = {Machine review of arXiv:2412.20612}
}
read the original abstract

Iterative Closest Point (ICP) is a commonly used algorithm to estimate transformation between two point clouds. The key idea of this work is to leverage recent advances in explainable AI for probabilistic ICP methods that provide uncertainty estimates. Concretely, we propose a method that can explain why a probabilistic ICP method produced a particular output. Our method is based on kernel SHAP (SHapley Additive exPlanations). With this, we assign an importance value to common sources of uncertainty in ICP such as sensor noise, occlusion, and ambiguous environments. The results of the experiment show that this explanation method can reasonably explain the uncertainty sources, providing a step towards robots that know when and why they failed in a human interpretable manner

Figures

Figures reproduced from arXiv: 2412.20612 by the authors.

Figure 1
Figure 1. The simplified input on the left, represented by binary conditions, are mapped to the feature [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. SHAP summary plot [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 4
Figure 4. SHAP dependence plots. ICP pose estimates of perturbed inputs. By comparing the covariance matrices of these two normal distributions via KL divergence, we estimate the uncertainty as a scalar, representing the combined effect of all three uncertainty sources. 6.1 Different Perturbations for the Same Pair of Input Point Clouds in Apartment In this section, we examine various perturbation sets to understand how kerne… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 22 canonical work pages

  1. [1]

    Izadi, D

    S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. Davison, et al. Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera. In Proceedings of the 24th annual ACM symposium on User interface software and technology, pages 559–568, 2011

    2011

  2. [2]

    Y . Xia, R. Ding, Z. Qin, G. Zhan, K. Zhou, L. Yang, H. Dong, and D. Cremers. Targo: Benchmarking target-driven object grasping under occlusions.arXiv preprint arXiv:2407.06168, 2024. 8

    arXiv 2024

  3. [3]

    Wu, J.-C

    M.-L. Wu, J.-C. Chien, C.-T. Wu, and J.-D. Lee. An augmented reality system using improved- iterative closest point algorithm for on-patient medical image visualization. Sensors (Basel, Switzerland), 18, 2018. URL https://api.semanticscholar.org/CorpusID:51909358

    work page 2018

  4. [4]

    P. J. Besl and N. D. McKay. Method for registration of 3-d shapes. In Sensor fusion IV: control paradigms and data structures, volume 1611, pages 586–606. Spie, 1992

    1992

  5. [5]

    A. Censi. An accurate closed-form estimate of icp’s covariance. In Proceedings 2007 IEEE international conference on robotics and automation, pages 3167–3172. IEEE, 2007

    work page 2007

  6. [6]

    F. A. Maken, F. Ramos, and L. Ott. Stein icp for uncertainty estimation in point cloud matching. IEEE Robotics and Automation Letters, 7(2):1063–1070, 2021

    work page 2021

  7. [7]

    Brossard, S

    M. Brossard, S. Bonnabel, and A. Barrau. A new approach to 3d icp covariance estimation. IEEE Robotics and Automation Letters, 5(2):744–751, 2020

    work page 2020

  8. [8]

    F. A. Maken, F. Ramos, and L. Ott. Estimating motion uncertainty with bayesian icp. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 8602–8608. IEEE, 2020

    work page 2020

Show all 27 references
  1. [9]

    Landry, F

    D. Landry, F. Pomerleau, and P. Giguere. Cello-3d: Estimating the covariance of icp in the real world. In 2019 International Conference on Robotics and Automation (ICRA), pages 8190–8196. IEEE, 2019

    2019

  2. [10]

    S. M. Lundberg and S.-I. Lee. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017

    2017

  3. [11]

    why should i trust you?

    M. T. Ribeiro, S. Singh, and C. Guestrin. "why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016

    2016

  4. [12]

    Lipovetsky and M

    S. Lipovetsky and M. Conklin. Analysis of regression in game theory approach. Applied Stochastic Models in Business and Industry, 17(4):319–330, 2001

    2001

  5. [13]

    Štrumbelj and I

    E. Štrumbelj and I. Kononenko. Explaining prediction models and individual predictions with feature contributions. Knowledge and information systems, 41:647–665, 2014

    2014

  6. [14]

    Datta, S

    A. Datta, S. Sen, and Y . Zick. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. In 2016 IEEE symposium on security and privacy (SP), pages 598–617. IEEE, 2016

    2016

  7. [15]

    Rusinkiewicz and M

    S. Rusinkiewicz and M. Levoy. Efficient variants of the icp algorithm. In Proceedings third international conference on 3-D digital imaging and modeling, pages 145–152. IEEE, 2001

    2001

  8. [16]

    Chen and G

    Y . Chen and G. Medioni. Object modelling by registration of multiple range images.Image and vision computing, 10(3):145–155, 1992

    1992

  9. [17]

    Z. Wang, Y . Liu, Q. Liao, H. Ye, M. Liu, and L. Wang. Characterization of a rs-lidar for 3d perception. In 2018 IEEE 8th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER), pages 564–569. IEEE, 2018

    2018

  10. [18]

    Pomerleau, A

    F. Pomerleau, A. Breitenmoser, M. Liu, F. Colas, and R. Siegwart. Noise characterization of depth sensors for surface inspections. In 2012 2nd International Conference on Applied Robotics for the Power Industry (CARPI), pages 16–21. IEEE, 2012

    2012

  11. [19]

    X. Chen, T. Läbe, A. Milioto, T. Röhling, O. Vysotska, A. Haag, J. Behley, and C. Stachniss. OverlapNet: Loop Closing for LiDAR-based SLAM. In Proceedings of Robotics: Science and Systems (RSS), 2020. 9

    2020

  12. [20]

    Pomerleau, F

    F. Pomerleau, F. Colas, R. Siegwart, et al. A review of point cloud registration algorithms for mobile robotics. Foundations and Trends® in Robotics, 4(1):1–104, 2015

    2015

  13. [21]

    Bourmaud, R

    G. Bourmaud, R. Mégret, M. Arnaudon, and A. Giremus. Continuous-discrete extended kalman filter on matrix lie groups using concentrated gaussian distributions. Journal of Mathematical Imaging and Vision, 51:209–228, 2015

    2015

  14. [22]

    Pomerleau, M

    F. Pomerleau, M. Liu, F. Colas, and R. Siegwart. Challenging data sets for point cloud registration algorithms. The International Journal of Robotics Research, 31(14):1705–1711, Dec. 2012

    2012

  15. [23]

    J. Lee, J. Feng, M. Humt, M. G. Müller, and R. Triebel. Trust your robots! predictive uncertainty estimation of neural networks with sparse gaussian processes. InConference on Robot Learning, pages 1168–1179. PMLR, 2022

    2022

  16. [24]

    J. Lee, M. Humt, J. Feng, and R. Triebel. Estimating model uncertainty of neural networks in sparse information form. In International Conference on Machine Learning, pages 5702–5713. PMLR, 2020

    2020

  17. [25]

    Schnaus, J

    D. Schnaus, J. Lee, D. Cremers, and R. Triebel. Learning expressive priors for generalization and uncertainty estimation in neural networks. In International Conference on Machine Learning, pages 30252–30284. PMLR, 2023

    2023

  18. [26]

    J. Lee, R. Balachandran, Y . S. Sarkisov, M. De Stefano, A. Coelho, K. Shinde, M. J. Kim, R. Triebel, and K. Kondak. Visual-inertial telepresence for aerial manipulation. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 1222–1229. IEEE, 2020

    2020

  19. [27]

    J. Lee, R. Radhakrishna Balachandran, K. Kondak, A. Coelho, M. De Stefano, M. Humt, J. Feng, T. Asfour, and R. Triebel. Virtual reality via object pose estimation and active learning: Realizing telepresence robots with aerial manipulation capabilities. Field Robotics, 3:323–36...

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.