Pith. sign in

REVIEW 4 major objections 4 minor 69 references

Single-Frame Point-Pixel Registration via Supervised Cross-Modal Feature Matching

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A detector-free matcher registers single-frame LiDAR to camera, beating multi-frame baselines on nuScenes.

desk verdict The KITTI result is a plausible engineering contribution, but the nuScenes comparison likely uses ground-truth extrinsics to build the input, so the headline claim does not hold as written. read the letter →

arxiv 2506.22784 v2 pith:55FL4XJY submitted 2025-06-28 cs.CV cs.AIcs.RO

Yu Han , Zhiwei Huang , Yanting Zhang , Fangjun Ding , Shen Cai , Xiaoyu Tang , Yanchao Dong , Rui Fan This is my paper · ORCID
classification cs.CVcs.AIcs.RO
keywords single-frameLiDARpoint-pixelregistrationDAR-cameraextrinsiccalibrationcross-modalfeaturematchingdetector-freerepeatabilityscoringintensityprojectionautonomousdriving
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that point-pixel registration between a single-frame LiDAR point cloud and a camera image can be solved by projecting the point cloud's intensity into a 2D image and matching it to the grayscale camera image with a detector-free attention network. The authors argue that previous approaches fail under sparse LiDAR because they encode the two modalities separately or require accumulated point clouds, while a projection plus global-context matching can bridge the gap directly. They add a learned repeatability score that acts as a soft visibility prior, downweighting matches in occluded, textureless, or low-intensity regions. If the reported numbers hold, the method would make LiDAR-camera extrinsic calibration and fusion practical in settings where multi-frame accumulation is impossible, such as real-time or dynamic operation. On KITTI it reports translation error 0.25 m, rotation error 0.86 degrees, and 98.75 percent success; on nuScenes it reports 0.65 m, 1.73 degrees, and 84.33 percent success using only single-frame LiDAR, outperforming methods that rely on accumulated clouds.

What carries the argument

The central object is the confidence matrix $S(i,j)=P^{c}(i,j)\cdot S_{\mathrm{rep}}(i)$, formed by multiplying the Dual-Softmax matching probability from the attention-enhanced features by a per-patch repeatability score regressed by an MLP with sigmoid activation. This product is what lets the network downweight the spurious uniform-similarity matches that softmax forces in sparse projections, and it is supervised by a binary cross-entropy loss whose ground truth comes from a depth-consistency check. The other load-bearing component is the intensity projection itself: projecting LiDAR reflectance rather than depth preserves surface-material cues, giving the two modalities common local structure before the LoFTR-style transformer reasons globally.

What would settle it

Run the voxel-point-to-pixel and cross-modality retrieval baselines on the same 300 nuScenes pairs used in this paper's evaluation and recompute mean translation error, rotation error, and accuracy with the same thresholds; if their results on this sample are close to the full-set numbers in Table I, the central claim is supported, while if the sample is much easier than the full set, the claim that single-frame matching beats accumulated-cloud methods would need to be retested.

Watch

Extended reading notes

Core claim

The central claim is that the classical projection-based approach, previously dismissed for sparse data, becomes state of the art when paired with detector-free cross-modal matching and a repeatability prior. The authors project raw LiDAR intensity into a virtual 2D image and feed it, together with a grayscale camera image, into a dual-backbone ResNet-FPN feature extractor followed by a LoFTR-style transformer. The enhanced features are matched by cosine similarity and Dual-Softmax, and each LiDAR patch is assigned a repeatability score, predicted by an MLP from the cross-attended features and supervised by a depth-consistency check; the final confidence is the product of the matching probability and the repeatability score. Coarse-to-fine refinement produces sub-pixel correspondences, and EPnP with RANSAC recovers the 6-DoF pose. The consequence asserted is that single-frame sparse point clouds, without temporal accumulation and without monocular depth priors, suffice for accurate registration and calibration.

Load-bearing premise

The nuScenes comparison is fair: the 300 test pairs used by the authors are representative of the full nuScenes test set, so the numbers for earlier methods, taken from their original papers, are directly comparable to the numbers reported here.

Editorial extensions

If this is right

  • On the KITTI Odometry benchmark, the method reports reducing translation error from 0.61 m (previous state of the art) to 0.25 m and rotation error from 2.89 degrees to 0.86 degrees, with accuracy rising to 98.75 percent.
  • On nuScenes, the method reports translation error 0.65 m and rotation error 1.73 degrees, surpassing methods that use accumulated point clouds even though it uses a single 32-line LiDAR frame.
  • Against a fine-tuned LoFTR baseline, the repeatability scoring lowers the failure rate of producing fewer than four correspondences from 0.54 percent to 0.18 percent.
  • The ablation shows the repeatability module matters most on the sparser nuScenes data, where removing it raises rotation error from 1.73 degrees to 2.03 degrees and lowers precision from 88.75 to 88.45.
  • The method stays competitive with direct pose-regression calibration methods on KITTI sequence 00 while using far less training data and providing explicit correspondences rather than black-box pose output.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the intensity projection generalizes across sensors, the same architecture could be retrained for radar range-azimuth maps or thermal cameras, where the two inputs share geometry but not appearance.
  • The method's success suggests that the bottleneck in sparse LiDAR-camera matching is not the projection step but the lack of a global-context matcher plus a way to mark unreliable evidence; other projection-based tasks, such as extrinsics with rolling-shutter cameras, could adopt the same product-of-probability-and-visibility design.
  • The nuScenes result depends on a 300-pair test sample; a direct re-evaluation of the voxel-point-to-pixel, cross-modality retrieval, and dense-correspondence baselines on exactly those pairs would tell whether the advantage over accumulation-based methods is systematic or sample-dependent.
  • Because repeatability is supervised by depth consistency, the method implicitly needs reasonably accurate known poses for training; extending to fully self-supervised online calibration would require a proxy for that signal.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a projection-based point-pixel registration method that converts a single-frame LiDAR point cloud into a virtual intensity image, feeds this image together with a grayscale camera image into a detector-free LoFTR-style attention matcher, and augments the matching confidence with a learned repeatability score. The final pose is obtained by EPnP with RANSAC. Experiments are reported on KITTI Odometry, nuScenes, and MIAS-LCEC-TF70, with claims of state-of-the-art performance, including on nuScenes using only single-frame LiDAR.

Significance. If the reported results were measured on a protocol comparable to the baselines, the paper would make a useful contribution: applying detector-free cross-modal matching to sparse single-frame LiDAR intensity projections is a sensible idea, and the repeatability score as a soft visibility prior is clearly motivated and well described. The method is simple, and the KITTI numbers in Table I are strikingly good. However, the manuscript as submitted does not establish the central SOTA claim because the test-time input construction is ambiguous and, for nuScenes, appears to use ground-truth extrinsics, and because several comparisons and ablations are internally inconsistent. The paper does not provide code or model releases, so the results cannot be independently checked beyond the text.

major comments (4)
  1. [Section III-A and Section IV-A-2] The test-time extrinsic used to create the LiDAR intensity image is never specified. Section III-A defines ILiDAR by projecting the point cloud onto the camera image plane, which requires the unknown rigid transformation T. For nuScenes, Section IV-A-2 states that pairs are generated using the official SDK and that 'the corresponding point cloud is projected into the image plane using a perspective view centered at the front camera' — i.e., the ground-truth calibration — and no test-time perturbation is described. Under this construction ILiDAR and Icam are already aligned, the Dual-Softmax correspondences sit near the identity, and EPnP simply recovers the known pose from projection and PnP noise. The reported nuScenes numbers in Table I therefore measure refinement around a known pose, not registration from an unknown relative pose, while VP2P and Bie et al. solve from unaligned data. The comparison in Table I is not apples-to-apples, and the same ambiguity affects KITTI, where a perturbation is described only for training (Section IV-A-1). The paper must either define a test-time initial guess T0 and perturb it consistently for all methods, or reframe the problem as extrinsic refinement and compare against refinement methods.
  2. [Section IV-A-2 and Table I] The nuScenes evaluation is not on a common protocol with the cited baselines. The paper samples 300 test pairs for its own method (Section IV-A-2), while the numbers for VP2P, Bie et al., and other baselines in Table I appear to be taken from their original papers, which were evaluated on the full nuScenes test set. No re-evaluation of those baselines on the same 300 pairs is provided, so the 300-pair subset may not be representative and could be substantially easier. This directly threatens the abstract's claim that the method 'outperforms prior approaches on nuScenes (even those relying on accumulated point clouds)'. Moreover, even accepting the numbers as reported, Table I shows Ours has Acc=84.33% on nuScenes, below VP2P's 88.33% and Bie et al.'s 89.47%; the text in Section IV-D-1 reports only the et/er improvements and omits this accuracy deficit, and the abstract's blanket 'outperforms' statement is contradicted by the paper's own table.
  3. [Table V and Section IV-F] The ablation study does not support the stated conclusions. On KITTI, the Full model has precision 95.20, the lowest among the four rows, while Baseline has 96.00; w/o Dual Backbone achieves et=0.22 and er=0.71, both better than Full's et=0.22 and er=0.74. The text says the full combination achieves the best overall performance, but the table does not show this on KITTI. On nuScenes, w/o Dual Backbone has lower et/er than Baseline but also lower precision, and w/o Repeat. Score has the same et=0.65 as Full. The row names are ambiguous (what exactly is 'Baseline (fine-tuned)'?), and the conclusion that both the dual backbone and the repeatability score contribute positively is not consistently supported. The authors should clarify the row definitions and report a single consistent metric set, or revise the claims.
  4. [Table IV and Abstract] The MIAS-LCEC-TF70 results in Table IV do not support the abstract's global 'state-of-the-art' claim. Overall, Ours + EPnP reports et=0.152 and er=0.239, which is worse than DVL (et=0.082, er=0.218) and MIAS-LCEC (et=0.070, er=0.214). The text in Section IV-E-2 appropriately says the result is 'competitive', but this is in tension with the unqualified SOTA statement in the abstract and in Section I-C. The claims should be restricted to the specific settings and metrics where the method is actually best.
minor comments (4)
  1. [Table V] There is a typo in the row label 'w/o Pepeat. Socre'; it should read 'w/o Repeat. Score'.
  2. [Section IV-A-1] The training/test split description is confusing: it first says 'we use sequences 00-08 for training and sequences 09-10 for testing', then says 'we use sequence 02 ... for training, and reserve the remaining sequences for testing'. Please state the final protocol unambiguously and describe which sequences are used for the Table I KITTI numbers.
  3. [Section IV-B-2] The metric description says samples with fewer than four correspondences are excluded from average error computation but counted as failures in accuracy; this should be stated explicitly in the table captions or in the text near Table I so that readers can interpret the different denominators.
  4. [Table I] The table lists 'vpc + CoFiNet' and similar baselines without defining 'vpc' in the text or caption; please add a definition and state whether these baselines use accumulated or single-frame point clouds on each dataset.

Circularity Check

1 steps flagged · score 6.0 of 10

nuScenes pose 'prediction' reduces to the calibration used to build the input ILiDAR; the claimed SOTA over accumulated-cloud baselines is not measured by the reported protocol.

  1. self definitional [Section IV-A-2 (nuScenes pair generation), combined with Section III-A (ILiDAR projection) and Section III-E (EPnP pose estimation)]
    "The image-point cloud pairs are generated using the official nuScenes SDK, where each image is taken from the current frame, and the corresponding point cloud is projected into the image plane using a perspective view centered at the front camera. [Sec. III-A:] 'we first project the intensity from the raw point cloud, producing a virtual intensity image ILiDAR'. [Sec. III-E:] 'we estimate the extrinsic transformation T = [R | t] using a PnP solver'."

    On nuScenes the test input ILiDAR is built by projecting the point cloud with the camera-centered ground-truth perspective, so the 3D-2D correspondences handed to EPnP are (p, proj(T_gt·p)). The matcher then sees two already-aligned images and finds near-identity correspondences, and EPnP returns the very extrinsic used to construct ILiDAR, up to projection noise. The reported et=0.65 m, er=1.73° thus quantify identity matching plus PnP residuals, not registration from an unknown offset. No perturbation is specified for nuScenes test pairs (contrast KITTI training's '+1 m, ±10°'), so the claimed edge over VP2P/Bie et al., which solve from unaligned data, rests on the input containing the answer; the 'prediction' reduces to the calibration baked into the input.

full rationale

The paper's core training pipeline is not circular: ground-truth matches and the repeatability label are derived from the same pose/depth geometry used to build supervision, which is standard supervised learning, not a prediction that equals its fit. The LoFTR-style architecture, Dual-Softmax, and EPnP are externally cited, and the KITTI comparisons are against externally measured baselines; the method's matching component has independent content. Minor self-citation exists (MIAS-LCEC, ref. [13], shares authorship), but it is not load-bearing for the central KITTI/nuScenes claims. The load-bearing circularity is the nuScenes evaluation: the input intensity image is created by projecting the point cloud with the camera-centered (ground-truth) perspective, leaving the network to match already-aligned images, so EPnP recovers the same extrinsic used to build the input; the reported errors measure identity-matching plus solver noise rather than single-frame registration from an unknown pose, making the comparison with unaligned-input baselines (VP2P, Bie et al.) unfair by construction. Independently, Table I's own accuracy column (Ours 84.33% vs VP2P 88.33% and Bie et al. 89.47% on nuScenes) contradicts the abstract's blanket claim; that is a correctness problem, not circularity, but corroborates that the nuScenes SOTA claim is unsupported. The KITTI results would still stand if a test-time perturbation is applied, so the circularity is partial.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical entities. It relies on several domain assumptions about intensity-image alignment, supervision availability, cross-modal feature correlation, and transfer from image-pair pretraining. The listed free parameters are manually chosen hyperparameters and subsampling choices that directly affect the reported performance.

free parameters (6)
  • confidence threshold theta_c = not reported
    Used in Eq. 4 to filter coarse matches; directly controls the number of correspondences fed to EPnP and hence pose accuracy.
  • refinement window size w = not reported
    Determines the local crop size in coarse-to-fine refinement (Section III-C).
  • training perturbation range = +1 m translation, +/-10 deg rotation
    Virtual camera perturbations used to generate misaligned training pairs (Section IV-A-1); the method's robustness is tuned to this range.
  • nuScenes training/test pair counts = 500 training, 300 testing
    Uniform sampling of pairs from nuScenes (Section IV-A-2); the choice of subset affects the reported metrics and comparability with prior work.
  • frame subsampling rates = every 4 frames train, every 5 frames test (KITTI)
    Data subsampling in Section IV-A-1; affects effective training set size and evaluation.
  • learning rates and epochs = 1e-4 initial, 1e-5 fine-tune; 10 epochs KITTI 02, 5 epochs per sequence, 10 epochs MIAS
    Training schedule in Section IV-C; not justified by validation curves.
assumptions (4)
  • domain assumption An identity transformation approximately aligns ILiDAR and Icam when the network sees them
    Section III-B states 'we assume an identity transformation between ILiDAR and Icam.' The matcher is then expected to recover correspondences despite the actual unknown pose; this is central to the projection-based matching design and is only approximately true under the training perturbations.
  • domain assumption Ground-truth camera poses and depth maps are available to construct supervision
    Supervision (Section III-D) projects LiDAR points into the image using known poses and depth maps to generate Mgt_c and repeatability labels. The method therefore requires accurate pose and depth during training.
  • domain assumption LiDAR intensity reflects material properties that correlate with grayscale image structure
    Section III-A argues intensity projection captures lane markings and surface details visible in grayscale; the entire matching signal depends on this cross-modal correlation.
  • domain assumption LoFTR pretrained on MegaDepth transfers to cross-modal LiDAR intensity and grayscale pairs
    Initialization from LoFTR pretrained weights (Section IV-C); the method inherits the assumption that image-pair matching priors generalize to this cross-modal setting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Single-Frame Point-Pixel Registration via Supervised Cross-Modal Feature Matching." pith.science (2026). https://pith.science/paper/55FL4XJY

@misc{pith2026250622784,
  author       = {Pith},
  title        = {Pith review of: Single-Frame Point-Pixel Registration via Supervised Cross-Modal Feature Matching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/55FL4XJY}},
  note         = {Machine review of arXiv:2506.22784}
}
read the original abstract

Point-pixel registration between LiDAR point clouds and camera images is a fundamental yet challenging task in autonomous driving and robotic perception. A key difficulty lies in the modality gap between unstructured point clouds and structured images, especially under sparse single-frame LiDAR settings. Existing methods typically extract features separately from point clouds and images, then rely on hand-crafted or learned matching strategies. This separate encoding fails to bridge the modality gap effectively, and more critically, these methods struggle with the sparsity and noise of single-frame LiDAR, often requiring point cloud accumulation or additional priors to improve reliability. Inspired by recent progress in detector-free matching paradigms, we revisit the projection-based approach and introduce the detector-free framework for direct point-pixel matching between LiDAR and camera views. To further enhance matching reliability, we introduce a repeatability scoring mechanism that acts as a soft visibility prior. This guides the network to suppress unreliable matches in regions with low intensity variation, improving robustness under sparse input. Extensive experiments on KITTI, nuScenes, and MIAS-LCEC-TF70 benchmarks demonstrate that our method achieves state-of-the-art performance, outperforming prior approaches on nuScenes (even those relying on accumulated point clouds), despite using only single-frame LiDAR.

Figures

Figures reproduced from arXiv: 2506.22784 by the authors.

Figure 1
Figure 1. Visualization result of our point-pixel registration in KITTI Odometry 09 sequence. The colored region is the data fusion area which demonstrates [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Visual comparison of correspondence estimation by MatchAnything [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Architecture of the Proposed Multi-Modal Feature Matching Framework. Given a 4D point cloud and a real 2D RGB image [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Dual-Softmax-only matching vs. repeatability scoring. Red boxes show [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Visual comparison of Image-to-Point Cloud registration results under KITTI Odometry dataset. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 47 canonical work pages

  1. [1]

    TiCoSS: Tightening the coupling between semantic segmentation and stereo matching within a joint learning framework,

    G. Tang et al. , “TiCoSS: Tightening the coupling between semantic segmentation and stereo matching within a joint learning framework,” IEEE Transactions on Automation Science and Engineering , 2025

    work page 2025

  2. [2]

    Lce-calib: automatic LiDAR-frame/event camera extrinsic calibration with a globally optimal solution,

    J. Jiao et al., “Lce-calib: automatic LiDAR-frame/event camera extrinsic calibration with a globally optimal solution,” IEEE/ASME Transactions on Mechatronics, vol. 28, no. 5, pp. 2988–2999, 2023

    work page 2023

  3. [3]

    Real-time metric-semantic mapping for autonomous navigation in outdoor environments,

    J. Jiao, R. Geng, Y . Li, R. Xin, B. Yang, J. Wu, L. Wang, M. Liu, R. Fan, and D. Kanoulas, “Real-time metric-semantic mapping for autonomous navigation in outdoor environments,” IEEE Transactions on Automation Science and Engineering , 2024

    work page 2024

  4. [4]

    Edge-assisted multi-robot visual-inertial slam with efficient communication,

    X. Liu, S. Wen, J. Zhao, T. Z. Qiu, and H. Zhang, “Edge-assisted multi-robot visual-inertial slam with efficient communication,” IEEE Transactions on Automation Science and Engineering , 2024

    work page 2024

  5. [5]

    Dsqnet: A deformable model-based supervised learning algorithm for grasping unknown occluded objects,

    S. Kim, T. Ahn, Y . Lee, J. Kim, M. Y . Wang, and F. C. Park, “Dsqnet: A deformable model-based supervised learning algorithm for grasping unknown occluded objects,” IEEE Transactions on Automation Science and Engineering, vol. 20, no. 3, pp. 1721–1734, 2022

    work page 2022

  6. [6]

    Superglue: Learning feature matching with graph neural networks,

    P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superglue: Learning feature matching with graph neural networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 4938–4947

    2020

  7. [7]

    Loftr: Detector- free local feature matching with transformers,

    J. Sun, Z. Shen, Y . Wang, H. Bao, and X. Zhou, “Loftr: Detector- free local feature matching with transformers,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 8922–8931. 12

    work page 2021

  8. [8]

    R2d2: Reliable and repeatable detector and descriptor,

    J. Revaud, C. De Souza, M. Humenberger, and P. Weinzaepfel, “R2d2: Reliable and repeatable detector and descriptor,” Advances in neural information processing systems , vol. 32, 2019

    work page 2019

Show all 69 references
  1. [9]

    D3feat: Joint learning of dense detection and description of 3d local features,

    X. Bai, Z. Luo, L. Zhou, H. Fu, L. Quan, and C.-L. Tai, “D3feat: Joint learning of dense detection and description of 3d local features,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 6359–6367

    2020

  2. [10]

    Usip: Unsupervised stable interest point detection from 3d point clouds,

    J. Li and G. H. Lee, “Usip: Unsupervised stable interest point detection from 3d point clouds,” in Proceedings of the IEEE/CVF international conference on computer vision , 2019, pp. 361–370

    2019

  3. [11]

    P2-net: Joint description and detection of local features for pixel and point matching,

    B. Wang, C. Chen, Z. Cui, J. Qin, C. X. Lu, Z. Yu, P. Zhao, Z. Dong, F. Zhu, N. Trigoni et al., “P2-net: Joint description and detection of local features for pixel and point matching,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 16 004–16 013

    2021

  4. [12]

    CRLF: Automatic calibration and refinement based on line feature for LiDAR and camera in road scenes,

    T. Ma et al. , “CRLF: Automatic calibration and refinement based on line feature for LiDAR and camera in road scenes,” arXiv preprint arXiv:2103.04558, 2021

    2021 arXiv

  5. [13]

    Online, target-free LiDAR-camera extrinsic calibration via cross-modal mask matching,

    Z. Huang et al., “Online, target-free LiDAR-camera extrinsic calibration via cross-modal mask matching,” IEEE Transactions on Intelligent Vehicles, pp. 1–12, 2024, DOI: 10.1109/TIV .2024.3456299

    2024

  6. [14]

    Ep n p: An accurate o (n) solution to the p n p problem,

    V . Lepetit, F. Moreno-Noguer, and P. Fua, “Ep n p: An accurate o (n) solution to the p n p problem,” International journal of computer vision , vol. 81, pp. 155–166, 2009

    2009

  7. [15]

    Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,

    M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395, 1981

    1981

  8. [16]

    Matchanything: Universal cross-modality image matching with large- scale pre-training,

    X. He, H. Yu, S. Peng, D. Tan, Z. Shen, H. Bao, and X. Zhou, “Matchanything: Universal cross-modality image matching with large- scale pre-training,” arXiv preprint arXiv:2501.07556 , 2025

    2025 arXiv

  9. [17]

    nuscenes: A multimodal dataset for autonomous driving,

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 11 621–11 631

    2020

  10. [18]

    Pointnet: Deep learning on point sets for 3d classification and segmentation,

    C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 652–660

    2017

  11. [19]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space,

    C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” Advances in neural information processing systems , vol. 30, 2017

    2017

  12. [20]

    Deepi2p: Image-to-point cloud registration via deep classification,

    J. Li and G. H. Lee, “Deepi2p: Image-to-point cloud registration via deep classification,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 15 960–15 969

    2021

  13. [21]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

    2016

  14. [22]

    Differen- tiable registration of images and lidar point clouds with voxelpoint-to- pixel matching,

    J. Zhou, B. Ma, W. Zhang, Y . Fang, Y .-S. Liu, and Z. Han, “Differen- tiable registration of images and lidar point clouds with voxelpoint-to- pixel matching,” Advances in Neural Information Processing Systems , vol. 36, pp. 51 166–51 177, 2023

    2023

  15. [23]

    Image-to-point registration via cross- modality correspondence retrieval,

    L. Bie, S. Li, and K. Cheng, “Image-to-point registration via cross- modality correspondence retrieval,” in Proceedings of the 2024 Interna- tional Conference on Multimedia Retrieval , 2024, pp. 266–274

    2024

  16. [24]

    From big to small: Multi-scale local planar guidance for monocular depth estimation,

    J. H. Lee, M.-K. Han, D. W. Ko, and I. H. Suh, “From big to small: Multi-scale local planar guidance for monocular depth estimation,”arXiv preprint arXiv:1907.10326, 2019

    1907 arXiv

  17. [25]

    Are we ready for autonomous driving? the KITTI vision benchmark suite,

    A. Geiger et al. , “Are we ready for autonomous driving? the KITTI vision benchmark suite,” in 2012 IEEE conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, 2012, pp. 3354–3361

    2012

  18. [26]

    Method for registration of 3-d shapes,

    P. J. Besl and N. D. McKay, “Method for registration of 3-d shapes,” in Sensor fusion IV: control paradigms and data structures , vol. 1611. Spie, 1992, pp. 586–606

    1992

  19. [27]

    Least-squares fitting of two 3-d point sets,

    K. S. Arun, T. S. Huang, and S. D. Blostein, “Least-squares fitting of two 3-d point sets,” IEEE Transactions on pattern analysis and machine intelligence, no. 5, pp. 698–700, 1987

    1987

  20. [28]

    Fully convolutional geometric fea- tures,

    C. Choy, J. Park, and V . Koltun, “Fully convolutional geometric fea- tures,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 8958–8966

    2019

  21. [29]

    3d semantic segmen- tation with submanifold sparse convolutional networks,

    B. Graham, M. Engelcke, and L. Van Der Maaten, “3d semantic segmen- tation with submanifold sparse convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 9224–9232

    2018

  22. [30]

    Kpconv: Flexible and deformable convolution for point clouds,

    H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, and L. J. Guibas, “Kpconv: Flexible and deformable convolution for point clouds,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 6411–6420

    2019

  23. [31]

    Preda- tor: Registration of 3d point clouds with low overlap,

    S. Huang, Z. Gojcic, M. Usvyatsov, A. Wieser, and K. Schindler, “Preda- tor: Registration of 3d point clouds with low overlap,” in Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition , 2021, pp. 4267–4276

    2021

  24. [32]

    Geometric transformer for fast and robust point cloud registration,

    Z. Qin, H. Yu, C. Wang, Y . Guo, Y . Peng, and K. Xu, “Geometric transformer for fast and robust point cloud registration,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 11 143–11 152

    2022

  25. [33]

    Generalized correspondence matching via flexible hi- erarchical refinement and patch descriptor distillation,

    Y . Han et al. , “Generalized correspondence matching via flexible hi- erarchical refinement and patch descriptor distillation,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 10 290–10 297

    2024

  26. [34]

    E3CM: Epipolar-constrained cascade correspondence matching,

    C. Zhou et al. , “E3CM: Epipolar-constrained cascade correspondence matching,” Neurocomputing, vol. 559, p. 126788, 2023

    2023

  27. [35]

    Dive deeper into rectifying homography for stereo camera online self-calibration,

    H. Zhao et al. , “Dive deeper into rectifying homography for stereo camera online self-calibration,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 14 479–14 485

    2024

  28. [36]

    Road surface 3D reconstruction based on dense subpixel disparity map estimation,

    R. Fan et al., “Road surface 3D reconstruction based on dense subpixel disparity map estimation,” IEEE Transactions on Image Processing , vol. 27, no. 6, pp. 3025–3035, 2018

    2018

  29. [37]

    Distinctive image features from scale-invariant keypoints,

    D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International journal of computer vision , vol. 60, pp. 91–110, 2004

    2004

  30. [38]

    Orb: An efficient alternative to sift or surf,

    E. Rublee, V . Rabaud, K. Konolige, and G. Bradski, “Orb: An efficient alternative to sift or surf,” in 2011 International conference on computer vision. Ieee, 2011, pp. 2564–2571

    2011

  31. [39]

    Bundle adjustment—a modern synthesis,

    B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon, “Bundle adjustment—a modern synthesis,” in International workshop on vision algorithms . Springer, 1999, pp. 298–372

    1999

  32. [40]

    Automatic registration of airborne LiDAR point cloud data and optical imagery depth map based on line and points features,

    F. Lv and K. Ren, “Automatic registration of airborne LiDAR point cloud data and optical imagery depth map based on line and points features,” Infrared Physics & Technology , vol. 71, pp. 457–463, 2015

    2015

  33. [41]

    Autocalibration of LiDAR and optical cameras via edge alignment,

    J. Castorena et al., “Autocalibration of LiDAR and optical cameras via edge alignment,” in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2016, pp. 2862–2866

    2016

  34. [42]

    Pixel-level extrinsic self calibration of high resolution LiDAR and camera in targetless environments,

    C. Yuan et al. , “Pixel-level extrinsic self calibration of high resolution LiDAR and camera in targetless environments,” IEEE Robotics and Automation Letters, vol. 6, no. 4, pp. 7517–7524, 2021

    2021

  35. [43]

    Automatic extrinsic calibration of vision and LiDAR by maximizing mutual information,

    G. Pandey et al., “Automatic extrinsic calibration of vision and LiDAR by maximizing mutual information,” Journal of Field Robotics , vol. 32, no. 5, pp. 696–722, 2015

    2015

  36. [44]

    Robust calibration of vehicle solid-state LiDAR-camera perception system using line-weighted correspondences in natural en- vironments,

    S. Tang et al., “Robust calibration of vehicle solid-state LiDAR-camera perception system using line-weighted correspondences in natural en- vironments,” IEEE Transactions on Intelligent Transportation Systems , 2023, DOI: 10.1109/TITS.2023.3328062

    2023

  37. [45]

    Automatic registration of panoramic image sequence and mobile laser scanning data using semantic features,

    J. Li et al. , “Automatic registration of panoramic image sequence and mobile laser scanning data using semantic features,” ISPRS Journal of Photogrammetry and Remote Sensing , vol. 136, pp. 41–57, 2018

    2018

  38. [46]

    Automatic registration of point cloud and panoramic images in urban scenes based on pole matching,

    Y . Wang et al. , “Automatic registration of point cloud and panoramic images in urban scenes based on pole matching,” International Journal of Applied Earth Observation and Geoinformation , vol. 115, p. 103083, 2022

    2022

  39. [47]

    Auto-calibration method using stop signs for urban au- tonomous driving applications,

    Y . Han et al., “Auto-calibration method using stop signs for urban au- tonomous driving applications,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 13 179–13 185

    2021

  40. [48]

    SE-Calib: Semantic edges based LiDAR-camera bore- sight online calibration in urban scenes,

    Y . Liao et al. , “SE-Calib: Semantic edges based LiDAR-camera bore- sight online calibration in urban scenes,” IEEE Transactions on Geo- science and Remote Sensing , 2023, DOI: 10.1109/TGRS.2023.3278024

    2023

  41. [49]

    Online camera-LiDAR calibration with sensor semantic information,

    Y . Zhu et al., “Online camera-LiDAR calibration with sensor semantic information,” in 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 4970–4976

    2020

  42. [50]

    General, single-shot, target-less, and automatic LiDAR- camera extrinsic calibration toolbox,

    K. Koide et al., “General, single-shot, target-less, and automatic LiDAR- camera extrinsic calibration toolbox,” in 2023 IEEE International Con- ference on Robotics and Automation (ICRA) , 2023, pp. 11 301–11 307

    2023

  43. [51]

    Faster Segment Anything: Towards lightweight sam for mobile applications,

    C. Zhang et al. , “Faster Segment Anything: Towards lightweight sam for mobile applications,” arXiv preprint arXiv:2306.14289 , 2023

    2023 arXiv

  44. [52]

    2d3d-matchnet: Learning to match keypoints across 2d image and 3d point cloud,

    M. Feng, S. Hu, M. H. Ang, and G. H. Lee, “2d3d-matchnet: Learning to match keypoints across 2d image and 3d point cloud,” in 2019 International Conference on Robotics and Automation (ICRA) . IEEE, 2019, pp. 4790–4796

    2019

  45. [53]

    Intrinsic shape signatures: A shape descriptor for 3d object recognition,

    Y . Zhong, “Intrinsic shape signatures: A shape descriptor for 3d object recognition,” in 2009 IEEE 12th international conference on computer vision workshops, ICCV Workshops . IEEE, 2009, pp. 689–696

    2009

  46. [54]

    Corri2p: Deep image-to-point cloud registration via dense correspondence,

    S. Ren, Y . Zeng, J. Hou, and X. Chen, “Corri2p: Deep image-to-point cloud registration via dense correspondence,” IEEE Transactions on 13 Circuits and Systems for Video Technology , vol. 33, no. 3, pp. 1198– 1208, 2022

    2022

  47. [55]

    DCPI-Depth: Explicitly infusing dense correspondence prior to unsupervised monocular depth estimation,

    M. Zhang et al., “DCPI-Depth: Explicitly infusing dense correspondence prior to unsupervised monocular depth estimation,” IEEE Transactions on Image Processing , 2025, DOI: 10.1109/TIP.2025.3581422

    2025

  48. [56]

    ViPOcc: Leveraging visual priors from vision foundation models for single-view 3D occupancy prediction,

    Y . Feng et al., “ViPOcc: Leveraging visual priors from vision foundation models for single-view 3D occupancy prediction,” in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) , vol. 39, no. 3, 2025, pp. 3004–3012

    2025

  49. [57]

    SCIPaD: Incorporating spatial clues into unsupervised pose-depth joint learning,

    Y . Feng et al. , “SCIPaD: Incorporating spatial clues into unsupervised pose-depth joint learning,” IEEE Transactions on Intelligent Vehicles , 2024, DOI: 10.1109/TIV .2024.3460868

    2024

  50. [58]

    Neighbourhood consensus networks,

    I. Rocco, M. Cimpoi, R. Arandjelovi ´c, A. Torii, T. Pajdla, and J. Sivic, “Neighbourhood consensus networks,” Advances in neural information processing systems, vol. 31, 2018

    2018

  51. [59]

    Cofinet: Reliable coarse- to-fine correspondences for robust pointcloud registration,

    H. Yu, F. Li, M. Saleh, B. Busam, and S. Ilic, “Cofinet: Reliable coarse- to-fine correspondences for robust pointcloud registration,” Advances in Neural Information Processing Systems , vol. 34, pp. 23 872–23 884, 2021

    2021

  52. [60]

    Geotransformer: Fast and robust point cloud registration with geometric transformer,

    Z. Qin, H. Yu, C. Wang, Y . Guo, Y . Peng, S. Ilic, D. Hu, and K. Xu, “Geotransformer: Fast and robust point cloud registration with geometric transformer,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 8, pp. 9806–9821, 2023

    2023

  53. [61]

    CalibRCNN: Calibrating camera and lidar by recur- rent convolutional neural network and geometric constraints,

    J. Shi et al. , “CalibRCNN: Calibrating camera and lidar by recur- rent convolutional neural network and geometric constraints,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 10 197–10 202

    2020

  54. [62]

    CalibDNN: multimodal sensor calibration for perception using deep neural networks,

    G. Zhao et al., “CalibDNN: multimodal sensor calibration for perception using deep neural networks,” in Signal Processing, Sensor/Information Fusion, and Target Recognition XXX , vol. 11756. SPIE, 2021, pp. 324–335

    2021

  55. [63]

    RegNet: Multimodal sensor registration using deep neural networks,

    N. Schneider et al., “RegNet: Multimodal sensor registration using deep neural networks,” in 2017 IEEE Intelligent Vehicles Symposium (IV) . IEEE, 2017, pp. 1803–1810

    2017

  56. [64]

    From chaos to calibration: A geometric mutual infor- mation approach to target-free camera LiDAR extrinsic calibration,

    J. Borer et al. , “From chaos to calibration: A geometric mutual infor- mation approach to target-free camera LiDAR extrinsic calibration,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024, pp. 8409–8418

    2024

  57. [65]

    LCCNet: LiDAR and camera self-calibration using cost volume network,

    X. Lv et al. , “LCCNet: LiDAR and camera self-calibration using cost volume network,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2021, pp. 2894– 2901

    2021

  58. [66]

    RGGNet: Tolerance aware LiDAR-camera online calibration with geometric deep learning and generative model,

    K. Yuan et al. , “RGGNet: Tolerance aware LiDAR-camera online calibration with geometric deep learning and generative model,” IEEE Robotics and Automation Letters , vol. 5, no. 4, pp. 6956–6963, 2020

    2020

  59. [67]

    CalibNet: Geometrically supervised extrinsic calibration using 3D spatial transformer networks,

    G. Iyer et al., “CalibNet: Geometrically supervised extrinsic calibration using 3D spatial transformer networks,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2018, pp. 1110–1117

    2018

  60. [68]

    Megadepth: Learning single-view depth predic- tion from internet photos,

    Z. Li and N. Snavely, “Megadepth: Learning single-view depth predic- tion from internet photos,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 2041–2050

    2018

  61. [69]

    Kitti-360: A novel dataset and bench- marks for urban scene understanding in 2d and 3d,

    Y . Liao, J. Xie, and A. Geiger, “Kitti-360: A novel dataset and bench- marks for urban scene understanding in 2d and 3d,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 3, pp. 3292– 3310, 2022

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.