REVIEW 3 major objections 4 minor 1 cited by
Learning Point Correspondences In Radar 3D Point Clouds For Radar-Inertial Odometry
T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A self-supervised transformer learns point correspondences in sparse FMCW radar point clouds and, plugged into an open-source radar-inertial odometry estimator, cuts position RMSE by over 14 percent on the authors' UAV flights and by over…
desk verdict A clean swap experiment and an honest disclosure of variance make this a solid subfield contribution, but the headline self-collected improvements are not statistically reliable; the Coloradar result is stronger than the stress-test note implies. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is an $(N+1)\times(N+1)$ correspondence-affinity matrix $G$: PointNet embeds each 3D point, two transformer sub-networks exchange contextual information across the two scans, and the dot product of the resulting embeddings fills $G$ with match-likelihood scores. The appended zero row and column give every unmatched point a reserved class label, so training becomes multi-label classification with cross-entropy on point indices. At inference, solving a Linear Sum Assignment problem on the non-padded submatrix, followed by acceptance and field-of-view thresholds, produces the output correspondences. The same LSA machinery, applied to Euclidean distances between points transformed by the motion-capture pose, generates the training labels without hand annotation.
What would settle it
Take a held-out sequence with correspondences verified against a dense LiDAR-registered reference and compute precision and recall of the learned matcher; the central claim would be falsified if the matcher's correct-match rate is no better than the default matcher's on the same points, because then the reported odometry gains would have to come from somewhere other than match quality.
Extended reading notes
Core claim
The central claim is that a learning-based correspondence predictor, trained self-supervised on radar scans, provides a substantially better data-association signal for radar-inertial odometry than the default non-learning matcher. The paper treats the index of every point as a class label, reserves index 0 for unmatched points, and optimizes a multi-label cross-entropy loss over the LSA-derived ground-truth match set. In controlled experiments where only the matching algorithm is swapped inside the radar-inertial odometry estimator, the approach reduces the average norm of position RMSE by 14.28 percent on the self-collected dataset and 19.01 percent on Coloradar; with Doppler disabled and only 3D point matches used in the EKF update, the average improvement is 70.38 percent. Attitude accuracy does not reliably improve, and the paper attributes the weaker attitude gains to the sensor's poor angular resolution.
Load-bearing premise
The training labels are generated by assuming the motion-capture pose is exact and that the Euclidean closest point in the transformed previous scan is the true correspondence; if that label generator is wrong for many points, the network learns a corrupted target and the downstream gains could vanish.
Editorial extensions
If this is right
- Low-cost, low-power FMCW radar can serve as a practical odometry sensor for small UAVs, because the learned matcher tolerates the sparse, noisy clouds these chips produce.
- The self-supervised label recipe (LSA on motion-capture-transformed Euclidean distances plus multi-label cross-entropy) extends to other sensors whose scans are too sparse for hand annotation or for SVD-based odometry training.
- Swapping the matching component of an existing radar-inertial odometry estimator is a drop-in change: all EKF parameters and other measurement sources stay unchanged, and only the matcher is replaced.
- Because the matches-only configuration improves by over 70 percent, learned 3D point correspondences can carry the estimator even when radar Doppler measurements are absent or unusable, at least under moderate motion.
- The reported unoptimized inference time of 0.0273 seconds per scan pair indicates that real-time onboard use is within reach with a faster implementation.
Reading between the lines
- The paper leaves untested whether the same weights transfer to other radar chips or mounting geometries; the field-of-view and acceptance thresholds are tuned for the specific sensor used, so a new deployment would need retuning or retraining.
- Because the training labels are only as good as the motion-capture pose and the Euclidean-nearest-neighbor assignment, the method's ceiling is set by that label generator; a testable extension would train on labels from bundle-adjusted radar-only odometry and see whether downstream accuracy improves further.
- The 70 percent matches-only result suggests a radar-only odometry mode without Doppler is plausible in gentle motion, even though the authors note that Coloradar's aggressive motion breaks this configuration for both methods.
- An implication the authors do not draw is that the same correspondence-affinity matrix could be reused for place recognition or loop closure in a radar SLAM system, not just for consecutive-scan odometry.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a learning-based framework for finding point correspondences between consecutive, sparse 3D point clouds from a consumer-grade FMCW SoC radar. The architecture combines PointNet embeddings with two transformer sub-networks that produce an affinity matrix, from which correspondences are extracted by solving a linear sum assignment (LSA) problem. The network is trained with a multi-label cross-entropy loss using ground-truth matches generated automatically from motion-capture transforms and LSA on Euclidean distances, avoiding manual annotation. The method is evaluated inside an open-source EKF-based radar-inertial odometry (RIO) framework, where the only change is the matching algorithm. The authors report average position RMSE improvements of 14.28% on a self-collected UAV dataset, 19.01% on five Coloradar sequences, and 70.38% when point matches are the only source of correction in the update step.
Significance. If the reported improvements are reliable, the paper advances radar-inertial odometry for low-cost, sparse radar sensors, which is a practically relevant problem. The controlled swap experiment is a genuine strength: the same RIO estimator is run with only the matching algorithm changed, giving a clean measurement of the matcher's influence. The use of an external dataset (Coloradar) and the release of code and data are also positive. However, the central quantitative claim is not yet established statistically: the headline percentages are averages over only five runs, with large variance, and the paper itself acknowledges limited statistical relevance. The label-generation procedure via LSA on motion-capture-transformed geometry also needs scrutiny. With additional statistical rigor and clarification of the inference thresholds, the contribution could become a solid building block for the community.
major comments (3)
- [Section IV-B, Tables I and II] The headline claims of 14.28%, 19.01%, and 70.38% average improvements are based on only five runs and are not shown to be statistically reliable. In the Doppler+matches configuration on the self-collected dataset, the learning-based matcher is worse on runs 3 and 4 (0.714 vs 0.704 m and 0.380 vs 0.338 m), and the 14.28% average is driven largely by run 5 (0.230 vs 0.473 m). In the matches-only configuration, run 5 (0.736 vs 5.088 m) dominates the 70.38% average. A paired test on n=5 does not reach conventional significance levels, and the Coloradar comparison, though more consistent, still has a large standard deviation relative to the mean difference. The authors themselves state that 'With a standard deviation much higher than the improvements, these results have limited statistical relevance.' To support the central claim, the paper should report per-sequence paired differences with confidence intervals or bootstrap estimates, and ideally evaluate on additional trajectories or sequences.
- [Section III-B, Eq. (4)] The training labels are generated by solving an LSA problem on Euclidean distances between points in the first scan transformed by the motion-capture pose into the second scan frame. This presupposes that the motion-capture transform is accurate and that geometric nearest-neighbor assignment identifies true radar correspondences, even though SoC radar point clouds contain ghost reflections and spurious points. The paper provides no validation of the label quality, such as the fraction of LSA assignments that are geometrically consistent with independent criteria or the number of points assigned to the zero class. If a large share of labels are arbitrary LSA pairings between unrelated points, the network learns a corrupted target, and the downstream RIO gains may not transfer to new environments. Please add label-quality statistics or an ablation that trains on a subset with cleaner labels.
- [Section III-B and Figure 3] The inference pipeline applies an 'experimentally chosen' acceptance threshold and a FOV threshold, but the paper does not report the threshold values, how they were selected, or whether they were tuned on the same validation trajectories used to compute the reported improvements. If the thresholds were tuned on the test data, the evaluation is optimistic and the claimed gains may not generalize. The authors should specify the thresholds, state explicitly which data were used to choose them, and ideally perform a sensitivity analysis.
minor comments (4)
- [Abstract and Section III-B] The method is described as 'self-supervised,' but the training labels come from an external motion-capture system via LSA on transformed geometry. This is better described as 'automatically labeled' or 'supervised with automatically generated labels' to avoid terminological confusion.
- [Section III-A] The paper states that point clouds are zero-padded to a fixed length and that a zero vector is prepended, but it does not say whether padded positions are masked in the transformer attention. As written, the architecture attends over padding tokens, which may introduce artifacts; please clarify the masking strategy.
- [Section IV-B] The comparison with the method of [29] is not a controlled comparison because the underlying estimator differs, and the authors acknowledge this. The paragraph may still mislead readers; consider moving it to a separate qualitative discussion or presenting it with the caveat more prominently.
- [Equation (4)] The notation in Eq. (4) is dense and hard to parse; please define all superscripts and subscripts explicitly, and consider separating the rotation and translation terms into a cleaner expression.
Circularity Check
No significant circularity: the learning target is external mocap-derived geometry via LSA, and the evaluation is a controlled matcher swap inside an open-source baseline, including on the public Coloradar dataset.
full rationale
The paper's central claim is that replacing the default non-learning matcher in an open-source EKF RIO framework with the proposed transformer matcher reduces position RMSE. The derivation chain for this claim is not circular. Ground-truth correspondences for training are generated by taking the motion-capture transform between consecutive radar frames, transforming the previous scan into the current frame, and solving an LSA problem on Euclidean distances (Eq. 4); this is an external geometric source, not a function of the network output or of the RIO result. The network is trained with a multi-label cross-entropy loss against these labels (Eq. 3), and at inference the LSA is solved on the predicted affinity matrix (Eq. 2); no equation assumes the claimed odometry improvement. The evaluation is a controlled comparison in which only the matching algorithm is exchanged while 'all other parameters and settings remain the same,' and it includes the public Coloradar dataset in addition to the authors' self-collected trajectories, so the central comparison is not an artifact of the training data. The paper's only self-citations are to its own prior RIO estimator and matcher ([12], [13], [21]) as the evaluation baseline; this is benchmarking against prior work, not a load-bearing citation that forces the conclusion. The paper itself flags a real limitation: for the Doppler+matches configuration, 'the standard deviation much higher than the improvements... limited statistical relevance,' and for attitude the benefit is negative. Statistical fragility and label noise from the LSA/mocap pipeline are correctness risks, but they are not circularity. The term 'self-supervised' is imprecise because supervision comes from mocap-derived labels, but that is a naming issue rather than a circular reduction. No prediction reduces by construction to a fitted parameter or to a self-cited uniqueness/ansatz theorem.
Assumptions & free parameters
free parameters (2)
- Correspondence acceptance threshold =
Not reported
- Field-of-view (FOV) threshold =
Sensor dependent, not specified
assumptions (4)
- domain assumption The motion-capture pose transformation between consecutive radar frames is sufficiently accurate to serve as ground truth for match labels (Eq. 4).
- domain assumption A one-to-one LSA assignment over Euclidean distances in the transformed scan identifies the correct point correspondences for FMCW SoC radar scans.
- domain assumption The EKF RIO framework from [12], [21] is an adequate evaluation platform, and swapping only the matcher isolates matching quality.
- ad hoc to paper Zero-padding all point clouds to a fixed length N plus a prepended class-0 token preserves the information needed for matching.
Cite this review
Pith. "Pith review of Learning Point Correspondences In Radar 3D Point Clouds For Radar-Inertial Odometry." pith.science (2026). https://pith.science/paper/T7G3IADZ
@misc{pith2026250618580,
author = {Pith},
title = {Pith review of: Learning Point Correspondences In Radar 3D Point Clouds For Radar-Inertial Odometry},
year = {2026},
howpublished = {\url{https://pith.science/paper/T7G3IADZ}},
note = {Machine review of arXiv:2506.18580}
}
read the original abstract
Using 3D point clouds in odometry estimation in robotics often requires finding a set of correspondences between points in subsequent scans. While there are established methods for point clouds of sufficient quality, state-of-the-art still struggles when this quality drops. Thus, this paper presents a novel learning-based framework for predicting robust point correspondences between pairs of noisy, sparse and unstructured 3D point clouds from a light-weight, low-power, inexpensive, consumer-grade System-on-Chip (SoC) Frequency Modulated Continuous Wave (FMCW) radar sensor. Our network is based on the transformer architecture which allows leveraging the attention mechanism to discover pairs of points in consecutive scans with the greatest mutual affinity. The proposed network is trained in a self-supervised way using set-based multi-label classification cross-entropy loss, where the ground-truth set of matches is found by solving the Linear Sum Assignment (LSA) optimization problem, which avoids tedious hand annotation of the training data. Additionally, posing the loss calculation as multi-label classification permits supervising on point correspondences directly instead of on odometry error, which is not feasible for sparse and noisy data from the SoC radar we use. We evaluate our method with an open-source state-of-the-art Radar-Inertial Odometry (RIO) framework in real-world Unmanned Aerial Vehicle (UAV) flights and with the widely used public Coloradar dataset. Evaluation shows that the proposed method improves the position estimation accuracy by over 14 % and 19 % on average, respectively. The open source code and datasets can be found here: https://github.com/aau-cns/radar_transformer.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
H-RINS: Hierarchical Tightly-coupled Radar-Inertial State Estimation via Smoothing and Mapping
Hierarchical dual factor graphs with continuous bias-and-covariance injection from a persistent full-state backend suppress radar-inertial drift while delivering real-time odometry.
Reference graph
Works this paper leans on
-
[1]
Millimeter-wave technology for automotive radar sensors in the 77 ghz frequency band,
J. Hasch, E. Topak, R. Schnabel, T. Zwick, R. Weigel, and C. Wald- schmidt, “Millimeter-wave technology for automotive radar sensors in the 77 ghz frequency band,”IEEE Transactions on Microwave Theory and Techniques, vol. 60, no. 3, pp. 845–860, 2012
work page 2012
-
[2]
Degradation resilient lidar- radar-inertial odometry,
M. Nissov, N. Khedekar, and K. Alexis, “Degradation resilient lidar- radar-inertial odometry,” in2024 IEEE International Conference on Robotics and Automation (ICRA), 2024, pp. 8587–8594
work page 2024
-
[3]
Radar-inertial odometry for closed-loop control of resource-constrained aerial plat- forms,
J. Michalczyk, M. Scheiber, R. Jung, and S. Weiss, “Radar-inertial odometry for closed-loop control of resource-constrained aerial plat- forms,” in2023 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), 2023, pp. 61–68
2023
-
[4]
A new wave in robotics: Survey on recent mmwave radar applications in robotics,
K. Harlow, H. Jang, T. D. Barfoot, A. Kim, and C. Heckman, “A new wave in robotics: Survey on recent mmwave radar applications in robotics,”IEEE Transactions on Robotics, vol. 40, pp. 4544–4560, 2024
2024
-
[5]
A novel radar point cloud gen- eration method for robot environment perception,
Y . Cheng, J. Su, M. Jiang, and Y . Liu, “A novel radar point cloud gen- eration method for robot environment perception,”IEEE Transactions on Robotics, vol. 38, no. 6, pp. 3754–3773, 2022
work page 2022
-
[6]
Radar inertial odometry with online calibration,
C. Doer and G. F. Trommer, “Radar inertial odometry with online calibration,” in2020 European Navigation Conference (ENC). IEEE, 2020, pp. 1–10
work page 2020
-
[7]
An ekf based approach to radar inertial odometry,
——, “An ekf based approach to radar inertial odometry,” in2020 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI). IEEE, 2020, pp. 152–159
work page 2020
-
[8]
Yaw aided radar inertial odometry using manhattan world assumptions,
——, “Yaw aided radar inertial odometry using manhattan world assumptions,” in2021 28th Saint Petersburg International Conference on Integrated Navigation Systems (ICINS), 2021, pp. 1–9
work page 2021
Show all 31 references
-
[9]
Radar-inertial ego-velocity estimation for visually degraded environments,
A. Kramer, C. Stahoviak, A. Santamaria-Navarro, A.-A. Agha- Mohammadi, and C. Heckman, “Radar-inertial ego-velocity estimation for visually degraded environments,” in2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 5739–5746
2020
-
[10]
Ekf-based radar-inertial odometry with online temporal calibration,
C. Kim, G. Bae, W. Shin, S. Wang, and H. Oh, “Ekf-based radar-inertial odometry with online temporal calibration,” 2025. [Online]. Available: https://arxiv.org/abs/2502.00661
2025 arXiv
-
[11]
3d ego-motion estimation using low-cost mmwave radars via radar velocity factor for pose-graph slam,
Y . S. Park, Y .-S. Shin, J. Kim, and A. Kim, “3d ego-motion estimation using low-cost mmwave radars via radar velocity factor for pose-graph slam,”IEEE Robotics and Automation Letters, vol. 6, no. 4, pp. 7691– 7698, 2021
2021
-
[12]
Multi-state tightly-coupled ekf-based radar-inertial odometry with persistent land- marks,
J. Michalczyk, R. Jung, C. Brommer, and S. Weiss, “Multi-state tightly-coupled ekf-based radar-inertial odometry with persistent land- marks,” in2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 4011–4017
2023
-
[13]
Tightly-coupled ekf-based radar-inertial odometry,
J. Michalczyk, R. Jung, and S. Weiss, “Tightly-coupled ekf-based radar-inertial odometry,” in2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 12 336–12 343
2022
-
[14]
Milli-rio: Ego-motion estimation with low-cost millimetre-wave radar,
Y . Almalioglu, M. Turan, C. X. Lu, N. Trigoni, and A. Markham, “Milli-rio: Ego-motion estimation with low-cost millimetre-wave radar,”IEEE Sensors Journal, vol. 21, no. 3, pp. 3314–3323, 2020
2020
-
[15]
Less is more: Physical-enhanced radar-inertial odometry,
Q. Huang, Y . Liang, Z. Qiao, S. Shen, and H. Yin, “Less is more: Physical-enhanced radar-inertial odometry,” in2024 IEEE Interna- tional Conference on Robotics and Automation (ICRA), 2024, pp. 15 966–15 972
2024
-
[16]
4d iriom: 4d imaging radar inertial odometry and mapping,
Y . Zhuang, B. Wang, J. Huai, and M. Li, “4d iriom: 4d imaging radar inertial odometry and mapping,”IEEE Robotics and Automation Letters, vol. 8, no. 6, pp. 3246–3253, 2023
2023
-
[17]
Precise ego-motion estimation with millimeter-wave radar under diverse and challenging conditions,
S. H. Cen and P. Newman, “Precise ego-motion estimation with millimeter-wave radar under diverse and challenging conditions,” in 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2018, pp. 6045–6052
2018
-
[18]
Under the radar: Learning to predict robust keypoints for odometry estimation and metric localisation in radar,
D. Barnes and I. Posner, “Under the radar: Learning to predict robust keypoints for odometry estimation and metric localisation in radar,” in2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 9484–9490
2020
-
[19]
Radar odometry combining probabilistic estimation and unsupervised feature learning,
K. Burnett, D. J. Yoon, A. P. Schoellig, and T. D. Barfoot, “Radar odometry combining probabilistic estimation and unsupervised feature learning,”arXiv preprint arXiv:2105.14152, 2021
2021 arXiv
-
[20]
Ardea—an mav with skills for future planetary missions,
P. Lutz, M. G. M ¨uller, M. Maier, S. Stoneman, T. Tomi ´c, I. von Bargen, M. J. Schuster, F. Steidle, A. Wedler, W. St ¨urzl, and R. Triebel, “Ardea—an mav with skills for future planetary missions,” Journal of Field Robotics, vol. 37, no. 4, pp. 515–551, 2020. [Online]. Avai...
2020 doi
-
[21]
Tightly-coupled factor graph formulation for radar-inertial odometry,
J. Michalczyk, J. Quell, F. Steidle, M. G. M ¨uller, and S. Weiss, “Tightly-coupled factor graph formulation for radar-inertial odometry,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024, pp. 3364–3370
2024
-
[22]
Deep radar detector,
D. Brodeski, I. Bilik, and R. Giryes, “Deep radar detector,” in2019 IEEE Radar Conference (RadarConf), 2019, pp. 1–6
2019
-
[23]
Ghost target detection in 3d radar data using point cloud based deep neural network,
M. Chamseddine, J. Rambach, D. Stricker, and O. Wasenmuller, “Ghost target detection in 3d radar data using point cloud based deep neural network,” in2020 25th International Conference on Pattern Recognition (ICPR), 2021, pp. 10 398–10 403
2021
-
[24]
Pct: Point cloud transformer,
M.-H. Guo, J.-X. Cai, Z.-N. Liu, T.-J. Mu, R. R. Martin, and S.-M. Hu, “Pct: Point cloud transformer,”Computational Visual Media, vol. 7, no. 2, p. 187–199, Apr. 2021. [Online]. Available: http://dx.doi.org/10.1007/s41095-021-0229-5
2021 doi
-
[25]
Deep closest point: Learning representa- tions for point cloud registration,
Y . Wang and J. Solomon, “Deep closest point: Learning representa- tions for point cloud registration,” in2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 3522–3531
2019
-
[26]
Pointnet: Deep learning on point sets for 3d classification and segmentation,
R. Q. Charles, H. Su, M. Kaichun, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 77–85
2017
-
[27]
Attention is all you need,
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” inAdvances in Neural Information Processing Systems, I. Guyon, U. V . Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnet...
2017
-
[28]
Coloradar: The direct 3d millimeter wave radar dataset,
A. Kramer, K. Harlow, C. Williams, and C. Heckman, “Coloradar: The direct 3d millimeter wave radar dataset,”The International Journal of Robotics Research, vol. 41, no. 4, pp. 351–360, 2022. [Online]. Available: https://doi.org/10.1177/02783649211068535
2022 doi
-
[29]
Incorporating point uncertainty in radar slam,
Y . Xu, Q. Huang, S. Shen, and H. Yin, “Incorporating point uncertainty in radar slam,”IEEE Robotics and Automation Letters, vol. 10, no. 3, pp. 2168–2175, 2025
2025
-
[30]
Do we need scan-matching in radar odometry?
V . Kubelka, E. Fritz, and M. Magnusson, “Do we need scan-matching in radar odometry?” in2024 IEEE International Conference on Robotics and Automation (ICRA), 2024, pp. 13 710–13 716
2024
-
[31]
Algorithms for the assignment and transportation prob- lems,
J. Munkres, “Algorithms for the assignment and transportation prob- lems,”Journal of the society for industrial and applied mathematics, vol. 5, no. 1, pp. 32–38, 1957
1957
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.