pith. sign in

arxiv: 2606.27818 · v1 · pith:AH7QLDAInew · submitted 2026-06-26 · 💻 cs.CV · cs.LG

Scalable and Differentiable Point-Cloud Registration Using Maximum Mean Discrepancy

Rixon Crane , Fahira Afzal Maken , Nicholas Lawrance , Stanislav Funiak , Kasra Khosoussi , Ming Xu , Russell Tsuchida This is my paper

Pith reviewed 2026-06-29 04:54 UTC · model grok-4.3

classification 💻 cs.CV cs.LG
keywords point cloud registrationmaximum mean discrepancyrandom Fourier featuresdifferentiable optimizationcorrespondence-freenonlinear least squaresset transformer
0
0 comments X

The pith

Point-cloud registration can be solved as a differentiable nonlinear least-squares problem using maximum mean discrepancy approximated by random Fourier features.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces MMD-Reg to register point clouds without establishing point correspondences. It formulates the problem using the maximum mean discrepancy between the two clouds, approximated efficiently with random Fourier features to achieve linear complexity. This objective is minimized with standard nonlinear solvers and made differentiable, allowing it to serve as a layer in larger neural networks. A sympathetic reader would care because this supports registration in difficult cases like bad initial guesses or partial overlaps, and enables end-to-end learning.

Core claim

Registration is modeled as a nonlinear least-squares problem based on the Maximum Mean Discrepancy between source and target point clouds, with the discrepancy approximated via random Fourier features. The resulting objective admits efficient solution by methods such as Levenberg-Marquardt and is differentiable with respect to the transformation parameters through the implicit function theorem.

What carries the argument

Maximum Mean Discrepancy approximated by random Fourier features, formulated as a nonlinear least-squares objective for rigid transformation estimation.

If this is right

  • The objective can be solved efficiently with Levenberg-Marquardt.
  • The solution is differentiable via the implicit function theorem.
  • MMD-Reg can be used as a differentiable optimization layer in end-to-end trainable models.
  • It supports registration under poor initial alignment and partial overlap.
  • The Neural MMD-Reg variant integrates with a set transformer for supervised and unsupervised training.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Such a layer could be inserted into pipelines for 3D reconstruction or robot localization to improve robustness.
  • Extending the approximation to other kernels might broaden applicability to non-rigid registration.
  • Comparing the method's convergence rate to iterative closest point variants on standard benchmarks would clarify its practical speed advantage.

Load-bearing premise

The random Fourier feature approximation to the maximum mean discrepancy remains accurate enough to yield reliable registration even when the initial alignment is poor or the overlap is only partial.

What would settle it

Running the method on benchmark datasets with deliberately poor initial transformations and measuring whether the final registration error exceeds that of correspondence-based baselines would falsify the claim if consistently worse.

Figures

Figures reproduced from arXiv: 2606.27818 by Fahira Afzal Maken, Kasra Khosoussi, Ming Xu, Nicholas Lawrance, Rixon Crane, Russell Tsuchida, Stanislav Funiak.

Figure 1
Figure 1. Figure 1: Set transformer architecture for Neural MMD-Reg. Alter￾nating CABs process source and target point clouds, followed by concatenation and set attention. PMAs predicts global parameters, with the scale parameter constrained via a softplus nonlinearity. weights are trained using a cross-entropy loss with ground￾truth overlap masks, and gradients with respect to ℓNet are again obtained via implicit differentia… view at source ↗
Figure 2
Figure 2. Figure 2: Average rotation error↓, translation error↓, and CPU/GPU runtime↓ versus the number of points on PCPNet. MMD-Reg shows gradual CPU scaling and near-constant GPU runtime with decreasing error, while correspondence-based (ICP, GICP) and probabilistic (FilterReg, GMMReg, SVR, CPD) baselines exhibit increasing runtimes with limited accuracy gains. In [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Registration examples on PCPNet under gradient density (top row), high noise (middle row), and 20% outlier (bottom row) settings. MMD-Reg (rightmost column) maintains accurate alignment across all conditions, ICP and GICP exhibit degraded performance in the presence of outliers when not augmented with robustness mechanisms. the fraction of points in both source and target clouds that are replaced by unifor… view at source ↗
Figure 4
Figure 4. Figure 4: Average rotation error↓ and translation error↓ versus out￾lier percentage on PCPNet. MMD-Reg exhibits greater robustness to increasing outlier ratios than competing approaches. pairs from the sequences K-03, K-04, V-03, and V-04. Pairs are formed by selecting frames separated by a stride of three, with the earlier and later frames treated as the source and target point clouds, respectively. All point cloud… view at source ↗
read the original abstract

We present MMD-Reg, a novel correspondence-free approach to point-cloud registration that is differentiable and has linear computational complexity in the number of points. We model registration as a nonlinear least-squares problem based on the Maximum Mean Discrepancy, approximated using random Fourier features. The resulting objective can be solved efficiently with standard methods such as Levenberg-Marquardt, and the solution is differentiable via the implicit function theorem. This allows MMD-Reg to be used as a differentiable optimization layer within end-to-end trainable models, supporting registration under challenging conditions such as poor initial alignment and partial overlap. We demonstrate this Neural MMD-Reg formulation by integrating the layer with a set transformer, training the resulting model in supervised and unsupervised settings, and comparing its performance against recent learning-based methods. We also evaluate standalone MMD-Reg, comparing its accuracy and scalability against widely used non-learning-based registration methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper presents MMD-Reg, a correspondence-free point-cloud registration method formulated as nonlinear least-squares minimization of the Maximum Mean Discrepancy (MMD) between source and target clouds. The MMD is approximated via random Fourier features (RFF) to achieve linear complexity in the number of points; the resulting objective is solved with Levenberg-Marquardt and differentiated end-to-end via the implicit function theorem. A neural variant (Neural MMD-Reg) is obtained by embedding the solver as a differentiable layer inside a set transformer, trained in both supervised and unsupervised regimes, and evaluated against recent learning-based and classical registration baselines under conditions including poor initialization and partial overlap.

Significance. If the RFF surrogate preserves the optimization landscape of the true MMD, the approach supplies a scalable, correspondence-free, and fully differentiable registration primitive that can be dropped into larger networks. This would be useful for end-to-end pipelines that must handle real-world point-cloud data without reliable correspondences or good initial poses.

major comments (2)
  1. [Modeling / approximation paragraphs] The central modeling claim (abstract and modeling section) treats the RFF approximation to MMD as interchangeable with the exact MMD for the purposes of Levenberg-Marquardt convergence and IFT differentiability. No explicit error bounds, concentration results, or sensitivity analysis are provided that quantify how the random approximation error (which depends on the number of features, kernel bandwidth, and point-set discrepancy) affects the location or depth of minima under poor initialization or partial overlap—the regimes explicitly advertised as supported.
  2. [Experiments section] The experimental validation of standalone MMD-Reg and Neural MMD-Reg reports accuracy and scalability gains, yet does not include controlled ablations that isolate the effect of RFF sample size on registration success rate when initial alignment is far from correct or overlap is incomplete. Without such controls it is impossible to confirm that the claimed reliability follows from the modeling choices rather than from favorable hyper-parameter settings or test-set bias.
minor comments (2)
  1. [Method] Notation for the RFF mapping and the resulting least-squares residual vector should be introduced once with explicit dimensions so that the subsequent Levenberg-Marquardt and IFT derivations are easier to follow.
  2. [Related work] The paper cites the standard RFF and MMD literature but does not discuss any recent work on kernel approximation error for registration or alignment tasks; adding one or two targeted references would strengthen the positioning.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the detailed and constructive feedback. We address each major comment below, indicating planned revisions where appropriate.

read point-by-point responses

  1. Referee: [Modeling / approximation paragraphs] The central modeling claim (abstract and modeling section) treats the RFF approximation to MMD as interchangeable with the exact MMD for the purposes of Levenberg-Marquardt convergence and IFT differentiability. No explicit error bounds, concentration results, or sensitivity analysis are provided that quantify how the random approximation error (which depends on the number of features, kernel bandwidth, and point-set discrepancy) affects the location or depth of minima under poor initialization or partial overlap—the regimes explicitly advertised as supported.

    Authors: We acknowledge that the manuscript does not provide new, registration-specific error bounds or sensitivity analysis for the RFF approximation. The RFF approximation to MMD is a standard technique whose concentration properties are established in the literature (Rahimi & Recht 2007; subsequent MMD estimation results). Our formulation relies on these existing guarantees to ensure the approximated objective remains amenable to LM and IFT. A dedicated theoretical analysis of how approximation error perturbs minima under partial overlap would require substantial new work outside the paper's scope. We will add a paragraph in the modeling section citing the relevant approximation guarantees and discussing their implications for the claimed regimes. revision: partial

  2. Referee: [Experiments section] The experimental validation of standalone MMD-Reg and Neural MMD-Reg reports accuracy and scalability gains, yet does not include controlled ablations that isolate the effect of RFF sample size on registration success rate when initial alignment is far from correct or overlap is incomplete. Without such controls it is impossible to confirm that the claimed reliability follows from the modeling choices rather than from favorable hyper-parameter settings or test-set bias.

    Authors: We agree that controlled ablations on RFF sample size would strengthen the experimental section. In the revised manuscript we will add experiments that vary the number of random Fourier features while holding other hyperparameters fixed, and report registration success rates specifically under poor initialization and partial-overlap conditions using the same protocols and datasets as the original evaluation. revision: yes

standing simulated objections not resolved
  • Deriving explicit error bounds or a full sensitivity analysis quantifying the impact of RFF approximation error on the location or depth of minima under poor initialization and partial overlap.

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The derivation models registration as MMD minimization (approximated by random Fourier features), solved via Levenberg-Marquardt and differentiated via the implicit function theorem. These steps draw on external kernel, approximation, and optimization literature without reducing any load-bearing claim to a self-defined quantity, fitted input renamed as prediction, or self-citation chain. The central objective and differentiability argument remain independent of the authors' prior work.

Axiom & Free-Parameter Ledger

2 free parameters · 2 axioms · 0 invented entities

The approach rests on standard kernel approximation and optimization results from prior literature rather than new invented entities; the main additions are the modeling choice and differentiability application.

free parameters (2)
  • number of random Fourier features
    Controls approximation quality and speed; must be chosen or tuned for the method to work as claimed.
  • MMD kernel bandwidth
    Affects the discrepancy measure and is typically selected per dataset or task.
axioms (2)
  • domain assumption Random Fourier features yield a sufficiently faithful approximation to the MMD kernel for registration purposes.
    Invoked when the abstract states the objective is approximated using random Fourier features.
  • standard math The implicit function theorem applies to the Levenberg-Marquardt solution of the nonlinear least-squares problem.
    Used to establish differentiability of the registration output.

pith-pipeline@v0.9.1-grok · 5707 in / 1599 out tokens · 39786 ms · 2026-06-29T04:54:29.662170+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

52 extracted references · 1 canonical work pages · 1 internal anchor

  1. [1]

    The simultaneous localization and mapping (

    Alsadik, Bashar and Karam, Samer , year =. The simultaneous localization and mapping (. Journal of Applied Science and Technology Trends , publisher =

  2. [2]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Aoki, Yasuhiro and Goforth, Hunter and Srivatsan, Rangaprasad Arun and Lucey, Simon , year =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

  3. [3]

    2019 , booktitle =

    Analysis of Robust Functions for Registration Algorithms , author =. 2019 , booktitle =

    2019

  4. [4]

    and McKay, Neil D

    Besl, P.J. and McKay, Neil D. , year =. A Method for Registration of. IEEE Transactions on Pattern Analysis and Machine Intelligence , publisher =

  5. [5]

    2022 , booktitle =

    Efficient and Modular Implicit Differentiation , author =. 2022 , booktitle =

    2022

  6. [6]

    2023 , journal =

    Point cloud registration: a mini-review of current state, challenging issues and future directions , author =. 2023 , journal =

    2023

  7. [7]

    IEEE Transactions on Visualization and Computer Graphics , publisher =

    Cao, Hualong and Wang, Yongcai and Li, Deying , year =. IEEE Transactions on Visualization and Computer Graphics , publisher =

  8. [8]

    Carlone, Luca and Kim, Ayoung and Barfoot, Timothy and Cremers, Daniel and Dellaert, Frank , year =

  9. [9]

    1992 , journal =

    Object modelling by registration of multiple range images , author =. 1992 , journal =

    1992

  10. [10]

    Journal of Computer-Aided Design & Computer Graphics , volume =

    Chen, Ruixing and Wu, Jun and Zhao, Xuemei and Xu, Gang and Luo, Ying and Wang, Haijian , year =. Journal of Computer-Aided Design & Computer Graphics , volume =

  11. [11]

    IEEE Robotics and Automation Letters , publisher =

    Cheng, Panfei and Yu, Hongshan and Liu, Changxiang and Luo, Kan and Akhtar, Naveed and Chen, Xieyuanli , year =. IEEE Robotics and Automation Letters , publisher =

  12. [12]

    and Laird, Nan M

    Dempster, Arthur P. and Laird, Nan M. and Rubin, Donald B. , year =. Maximum Likelihood from Incomplete Data Via the. Journal of the Royal Statistical Society: Series B (Methodological) , publisher =

  13. [13]

    Sensors , publisher =

    Dong, Jiahui and Tian, Maoyi and Yu, Jiayong and Li, Guoyu and Wang, Yunfei and Su, Yuxin , year =. Sensors , publisher =

  14. [14]

    2021 , booktitle =

    Robust Point Cloud Registration Framework Based on Deep Graph Matching , author =. 2021 , booktitle =

    2021

  15. [15]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Gao, Wei and Tedrake, Russ , year =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

  16. [16]

    Are we ready for Autonomous Driving? The

    Geiger, Andreas and Lenz, Philip and Urtasun, Raquel , year =. Are we ready for Autonomous Driving? The. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

  17. [17]

    New algorithms for

    Gold, Steven and Rangarajan, Anand and Lu, Chien-Ping and Pappu, Suguna and Mjolsness, Eric , year =. New algorithms for. Pattern Recognition , publisher =

  18. [18]

    2022 , journal =

    Deep Declarative Networks , author =. 2022 , journal =

    2022

  19. [19]

    2012 , journal =

    A Kernel Two-Sample Test , author =. 2012 , journal =

    2012

  20. [20]

    Predator: Registration of

    Huang, Shengyu and Gojcic, Zan and Usvyatsov, Mikhail and Wieser, Andreas and Schindler, Konrad , year =. Predator: Registration of. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

  21. [21]

    2011 , journal =

    Robust Point Set Registration Using Gaussian Mixture Models , author =. 2011 , journal =

    2011

  22. [22]

    , year =

    Kim, Pileun and Chen, Jingdao and Cho, Yong K. , year =. Automation in Construction , publisher =

  23. [23]

    Voxelized

    Koide, Kenji and Yokozuka, Masashi and Oishi, Shuji and Banno, Atsuhiko , year =. Voxelized. 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages =

    2021

  24. [24]

    2003 , publisher =

    The Implicit Function Theorem: History, Theory, and Applications , author =. 2003 , publisher =

    2003

  25. [25]

    2019 , booktitle =

    Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks , author =. 2019 , booktitle =

    2019

  26. [26]

    Mor. The. 1978 , booktitle =

    1978

  27. [27]

    2010 , journal =

    Point Set Registration: Coherent Point Drift , author =. 2010 , journal =

    2010

  28. [28]

    Parallel Tree Traversal for Nearest Neighbor Query on the

    Nam, Moohyeon and Kim, Jinwoong and Nam, Beomseok , year =. Parallel Tree Traversal for Nearest Neighbor Query on the. 2016 45th International Conference on Parallel Processing (ICPP) , pages =

    2016

  29. [29]

    2024 , journal =

    Deep Correspondence Matching-Based Robust Point Cloud Registration of Profiled Parts , author =. 2024 , journal =

    2024

  30. [30]

    2022 , booktitle =

    Theseus: A Library for Differentiable Nonlinear Optimization , author =. 2022 , booktitle =

    2022

  31. [31]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , publisher =

    Qin, Zheng and Yu, Hao and Wang, Changjian and Guo, Yulan and Peng, Yuxing and Ilic, Slobodan and Hu, Dewen and Xu, Kai , year =. IEEE Transactions on Pattern Analysis and Machine Intelligence , publisher =

  32. [32]

    2007 , journal =

    Random Features for Large-Scale Kernel Machines , author =. 2007 , journal =

    2007

  33. [33]

    Journal of Machine Learning Research , volume =

    Ren, Jie and Feng, Xidong and Liu, Bo and Pan, Xuehai and Fu, Yao and Mai, Luo and Yang, Yaodong , year =. Journal of Machine Learning Research , volume =

  34. [34]

    Efficient variants of the

    Rusinkiewicz, Szymon and Levoy, Marc , year =. Efficient variants of the. Proceedings Third International Conference on 3-D Digital Imaging and Modeling , pages =

  35. [35]

    Generalized-

    Segal, Aleksandr and Haehnel, Dirk and Thrun, Sebastian , year =. Generalized-. Proceedings of Robotics: Science and Systems , doi =

  36. [36]

    On the Error of Random

    Sutherland, Danica J and Schneider, Jeff , year =. On the Error of Random. Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence , pages =

  37. [37]

    2019 , booktitle =

    Deep Closest Point: Learning Representations for Point Cloud Registration , author =. 2019 , booktitle =

    2019

  38. [38]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Wu, Zhirong and Song, Shuran and Khosla, Aditya and Yu, Fisher and Zhang, Linguang and Tang, Xiaoou and Xiao, Jianxiong , year =. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

  39. [39]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , publisher =

    Yang, Jiaolong and Li, Hongdong and Campbell, Dylan and Jia, Yunde , year =. IEEE Transactions on Pattern Analysis and Machine Intelligence , publisher =

  40. [40]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Yew, Zi Jian and Lee, Gim Hee , year =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

  41. [41]

    Advances in Neural Information Processing Systems , volume =

    Yu, Hao and Li, Fu and Saleh, Mahdi and Busam, Benjamin and Ilic, Slobodan , year =. Advances in Neural Information Processing Systems , volume =

  42. [42]

    Research on the Improved

    Yuan, Honglei and Li, Guangyun and Wang, Li and Li, Xiangfei , year =. Research on the Improved. Sensors , publisher =

  43. [43]

    2016 , booktitle =

    Fast Global Registration , author =. 2016 , booktitle =

    2016

  44. [44]

    Exploring the Impact of Different Registration Methods and Noise Removal on the Registration Quality of Point Cloud Models in the Built Environment: A Case Study on

    Zhu, Zicheng and Rowlinson, Steve and Chen, Tianzhuo and Patching, Alan , year =. Exploring the Impact of Different Registration Methods and Noise Removal on the Registration Quality of Point Cloud Models in the Built Environment: A Case Study on. Buildings , publisher =

  45. [45]

    Present and future of

    Ebadi, Kamak and Bernreiter, Lukas and Biggie, Harel and Catt, Gavin and Chang, Yun and Chatterjee, Arghya and Denniston, Christopher E and Desch. Present and future of. IEEE Transactions on Robotics , volume=. 2023 , publisher=

    2023

  46. [46]

    2023 IEEE International Conference on Robotics and Automation (ICRA) , pages=

    Knights, Joshua and Vidanapathirana, Kavisha and Ramezani, Milad and Sridharan, Sridha and Fookes, Clinton and Moghadam, Peyman , year=. 2023 IEEE International Conference on Robotics and Automation (ICRA) , pages=

    2023

  47. [47]

    Open3D: A Modern Library for 3D Data Processing

    Zhou, Qian-Yi and Park, Jaesik and Koltun, Vladlen , journal=. 1801.09847 , archivePrefix=

    work page internal anchor Pith review Pith/arXiv arXiv

  48. [48]

    2015 , booktitle=

    An Adaptive Data Representation for Robust Point-Set Registration and Merging , author=. 2015 , booktitle=

    2015

  49. [49]

    , booktitle=

    Guerrero, Paul and Kleiman, Yanir and Ovsjanikov, Maks and Mitra, Niloy J. , booktitle=. 2018 , doi =

    2018

  50. [50]

    Journal of the Royal Statistical Society: Series B (Methodological) , volume=

    A general class of coefficients of divergence of one distribution from another , author=. Journal of the Royal Statistical Society: Series B (Methodological) , volume=. 1966 , publisher=

    1966

  51. [51]

    Density Ratio Estimation in Machine Learning , publisher=

    Sugiyama, Masashi and Suzuki, Taiji and Kanamori, Takafumi , year=. Density Ratio Estimation in Machine Learning , publisher=

  52. [52]

    Foundations and Trends in Robotics , volume=

    A review of point cloud registration algorithms for mobile robotics , author=. Foundations and Trends in Robotics , volume=. 2015 , publisher=

    2015