REVIEW 3 major objections 4 minor 96 references
X-Lens establishes that a 0.04B-parameter network can estimate metric depth from mixed fisheye and pinhole views, cutting AbsRel by 25.4% over the strongest baseline on a six-view heterogeneous benchmark while running at 22 FPS.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 06:08 UTC pith:JDDQKXCN
load-bearing objection Solid engineering and a big synthetic dataset, but the heterogeneous-camera claim is built on in-distribution synthetic numbers and zero-shot baselines. the 3 major comments →
X-Lens: Real-Time Metric Depth Estimation with Heterogeneous Cameras
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
X-Lens is a feed-forward network that takes a variable number of calibrated fisheye and pinhole views and predicts, per view, a dense normalized depth map, a confidence map, and a single global metric scalar; multiplying the scalar by the normalized depth yields metric depth. All internal geometry is expressed in ray space through a generic unprojection map that absorbs both pinhole intrinsics and fisheye distortion, avoiding panoramic resampling. Two modules carry the heterogeneous alignment: multi-view calibration tokens, injected layer-wise only for fisheye views, provide view-local lens correction, and a Jacobian-parameterized distortion bias, added to cross-view attention logits, favors
What carries the argument
The central object is the generic unprojection map G, which maps each pixel to a unit ray while absorbing per-camera calibration and camera type; every attention and positional operation then runs in this shared ray space rather than in pixel or pinhole-camera coordinates. Two mechanisms carry the heterogeneous-camera argument: (1) calibration tokens, learnable per network layer and per camera type, injected only for fisheye views and kept view-local in cross-attention, so lens distortion is absorbed without contaminating cross-view fusion; (2) a Jacobian distortion bias, computed from per-patch ray fields and their finite-difference Jacobians, added to cross-view attention logits to bias ma
Load-bearing premise
The headline heterogeneous-camera advantage rests on OmniScene-Full, a synthetic benchmark rendered with the same six-camera rig geometry, asset pipeline, and trajectory sampler that generated the model's training data, so the result may reflect in-distribution familiarity rather than genuine cross-lens generalization.
What would settle it
Take a real mixed fisheye-pinhole rig (roughly four 180-degree fisheye and two pinhole cameras) with LiDAR ground truth and intrinsics that differ from OmniScene's, and evaluate X-Lens zero-shot without any fine-tuning. If AbsRel degrades to near or above the level of strong monocular any-camera baselines, or if removing the Jacobian distortion bias makes no measurable difference, the claim that the geometry modules drive heterogeneous accuracy would be falsified.
If this is right
- If the central claim is correct, real-time metric depth on mixed fisheye-pinhole rigs is achievable with a 0.04B-parameter model, eliminating the need for panorama stitching or separate networks per camera type.
- The 25.4% AbsRel improvement over the strongest baseline on OmniScene-Full, with 88.9% fewer parameters, suggests the geometry-aware modules add measurable accuracy rather than relying on scale.
- The three-stage training recipe (pinhole pre-train, fisheye token adaptation, heterogeneous joint fine-tune) provides a transferable schedule for other heterogeneous sensor-fusion problems.
- Because all reasoning is in ray space, the same architecture can ingest pinhole, fisheye, and 360-degree cameras without architectural changes, as the paper states.
- The released OmniScene dataset, with scene-disjoint splits, provides a benchmark and training resource for future heterogeneous-camera depth research.
Where Pith is reading between the lines
- A natural next test, left implicit by the paper, is zero-shot evaluation on a real mixed fisheye-pinhole rig with LiDAR ground truth; if transfer holds, X-Lens could serve as a cheap geometric encoder for robot policies, a direction the appendix begins to explore.
- The Jacobian-bias idea is generic: any multi-camera transformer that knows pixel-to-ray mappings could inject similar local projection priors, potentially improving tasks such as occupancy prediction or 3D detection on heterogeneous rigs.
- The heaviest dependency is the synthetic-to-real bridge; the paper's own limitation section admits a sim-to-real gap for extreme FOV, so the headline gain may shrink outside OmniScene-like intrinsics.
- A testable extension is to ablate the calibration tokens and Jacobian bias on a real out-of-domain rig; if their contribution vanishes there, the gain may reflect training-distribution overlap rather than geometric conditioning.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents X-Lens, a 0.04B-parameter feed-forward model for metric depth estimation from a variable number of calibrated fisheye and pinhole views. The model combines a DINOv2/DPT backbone with three main components: layer- and camera-type-specific calibration tokens, a Jacobian-parameterized distortion bias injected into cross-view attention, and a confidence-weighted 'Scale Attention' pooled metric-scale head. Training is staged: (1) pinhole multi-view pretraining on 13 public datasets, (2) fisheye adaptation using only calibration tokens on the new OmniScene synthetic dataset, and (3) joint heterogeneous fine-tuning on OmniScene, KITTI360, and pinhole data. OmniScene contains about 266K six-view frames (four 180° fisheye plus two pinhole cameras) rendered from 103 Kujiale/UE scenes. Experiments cover fisheye-only (KITTI360, OmniScene-Single/Quad), pinhole-only (ETH3D, ScanNet++V2, OmniOcc), and heterogeneous (OmniScene-Full) settings. The headline claim is that X-Lens reduces AbsRel by 25.4% on OmniScene-Full over the strongest baseline (UniDAC) while using 88.9% fewer parameters and running at 22 FPS. The paper also reports competitive fisheye results and 41 FPS on KITTI360, plus a small downstream robotic-manipulation study.
Significance. If the heterogeneous-camera claims hold beyond the exact training distribution, X-Lens would be a practically valuable contribution: a compact, real-time model with explicit metric scale, no auxiliary reconstruction heads, and native support for mixed fisheye/pinhole rigs. The method is presented clearly, the loss in Eq. (7) directly supervises the metric scale through the |m-hat - m|/m term, so there is no evident circularity, and the staged training design is well motivated. The ablation study in Table 4 is thorough and internally consistent, showing that each stage and each proposed module contributes within the evaluated distribution. The release of code and the OmniScene dataset is a useful resource. However, the central 'superior heterogeneous-camera metric depth accuracy' claim is currently supported only by an in-distribution synthetic benchmark against zero-shot baselines, and the paper's own limitations section concedes a sim-to-real gap for unseen fisheye/FOV configurations. The significance of the work therefore depends on additional matched-training or real-mixed-rig evidence.
major comments (3)
- [§5.2, Table 3; §4; §D] The central heterogeneous-camera claim is measured only on OmniScene-Full. The test scenes are scene-disjoint, but the test split uses the same rendering assets (Kujiale/UE), the same Kannala-Brandt 180° fisheye model, the same pinhole intrinsics, the same six-camera rig, and the same occupancy-waypoint trajectory sampler as the training split. The compared baselines (UniDAC, DepthAnyCamera, MapAnything) are applied zero-shot, with monocular baselines applied per view without cross-view fusion, while X-Lens is trained on OmniScene in Stages 2 and 3. The 25.4% AbsRel reduction (0.1368→0.1021) and 68.1% Scale AbsRel reduction may therefore largely reflect train-test distribution overlap and evaluation asymmetry rather than the calibration-token/Jacobian-bias design. The internal ablations in Table 4 show the modules help within this distribution, but they do not control for data access. Pl
- [§5.1, Table 2 (ETH3D)] On real pinhole ETH3D, X-Lens achieves AbsRel 0.0445 and δ1 0.9723, compared with MapAnything at 0.0228/0.9992 and VGGT-Omega at 0.0055/0.9994. This is a large gap on a real-world benchmark and contradicts the abstract's 'competitive performance on conventional pinhole-only settings.' The paper reports only Scale AbsRel as the headline there (0.1217, which is best), but the dense-depth metrics are substantially worse than strong pinhole baselines. This result should be honestly contextualized; as written, it undermines the claim that X-Lens preserves pinhole performance and raises concerns about zero-shot generalization beyond the synthetic training distribution.
- [§7 (Limitations)] The Limitations section explicitly states that for 'unseen fisheye lens models with extreme FOV configurations that deviate drastically from the training distribution, a noticeable sim-to-real gap remains,' and the model strictly requires ground-truth calibration. There is no real-world mixed fisheye+pinhole evaluation in the paper: KITTI360 is fisheye-only, OmniOcc is pinhole-only, and OmniScene-Full is synthetic. Since the paper's central claim is 'superior heterogeneous-camera metric depth accuracy,' the absence of any real mixed-rig test means the headline claim is unsupported outside the synthetic training distribution. At minimum, add a small real mixed-rig evaluation or explicitly reframe the headline as an in-domain synthetic result.
minor comments (4)
- [§5.2, Table 3] The comparison with MapAnything claims '96.7% fewer parameters,' but MapAnything is a general feed-forward reconstruction model solving a broader task (joint depth, point maps, poses) than X-Lens. Parameter-count comparisons across models with different output scopes should be accompanied by a task-scope qualifier.
- [§5, Implementation Details] The FPS numbers are measured on a single H100 without specifying batch size, memory, or whether the reported FPS includes preprocessing/calibration-token injection. Please provide a reproducible speed protocol.
- [§6, Table 4] The 'Stage-2 only' row for pinhole OmniOcc is identical to 'Stage-1 only' (0.1052/0.0717/0.2801). This is expected since Stage 2 uses only fisheye data, but it should be stated explicitly to avoid confusion.
- [§7, Limitations] The wording 'noticeable sim-to-real gap remains, causing a slight degradation in performance' is internally inconsistent: a 'noticeable gap' is hard to reconcile with 'slight degradation.' Please align the language with the actual evidence.
Circularity Check
No significant circularity; the central depth/scale derivation is directly supervised and the evaluation-protocol concerns do not reduce to a derivation-by-construction.
full rationale
The paper's claimed derivation chain is self-contained. Metric scale is a directly supervised output: the total loss in Eq. (7) includes the term λ_scale |m̂−m|/m, and normalized depth is supervised in Eq. (8) against mean-normalized ground truth; no predicted quantity is defined in terms of its own evaluation target or fitted from the reported error metrics. The Jacobian distortion bias and calibration tokens are computed from the input calibration ray field by Eqs. (1), (3)–(5), not from depth ground truth. The OmniScene-Full headline result is obtained on a held-out, scene-disjoint split of the synthetic dataset used during training, which is a legitimate if limited evaluation design; the in-distribution nature of the test split and the zero-shot baseline setup are benchmark-comparability concerns, not circular reductions. The paper's own Limitations section concedes sim-to-real gaps for extreme unseen fisheye configurations, which further supports reading this as an external-generalization limitation rather than a derivational circularity. The only overlapping-author citation that appears (VO-DP [48]) is used in an auxiliary downstream appendix as a baseline, not as a load-bearing justification for the central depth-accuracy claim. No self-citation uniqueness theorem, ansatz-importation chain, or fitted-parameter-renamed-as-prediction is present.
Axiom & Free-Parameter Ledger
free parameters (4)
- Loss weights lambda_depth, lambda_grad, lambda_local, lambda_scale, lambda_mask, lambda_conf =
1.0, 1.0, 0.5, 1.0, 0.2, 1.0
- Calibration token count per layer K =
16
- Scale Attention confidence discard ratio =
25%
- Stage-3 view-count sampling range =
[6,8]
axioms (4)
- domain assumption Ground-truth camera calibration (intrinsics, extrinsics, camera type) is available for every input view.
- domain assumption OmniScene fisheye projections follow the Kannala-Brandt model with a fixed 180-degree FOV and a single radial profile.
- domain assumption DINOv2/DPT backbones supply transferable image features for metric depth regression.
- domain assumption A single global scalar m-hat suffices to convert normalized depth to metric depth for the whole rig.
invented entities (4)
-
OmniScene dataset
no independent evidence
-
Jacobian Distortion Bias
no independent evidence
-
Multi-View Calibration Tokens
no independent evidence
-
Scale Attention
no independent evidence
read the original abstract
We present X-lens, a compact feed-forward model for metric depth estimation from a variable number of calibrated fisheye and pinhole views. To support real-time downstream perception, X-lens is built around a geometry-aware heterogeneous camera formulation with two key components. Learnable calibration tokens provide a coarse alignment between fisheye and pinhole projective spaces, while a Jacobian-parameterized distortion bias injected into cross-attention models local projection changes and promotes cross-camera consistency, enabling robust generalization with only 0.04B parameters and up to 41 FPS. The model predicts dense depth together with a global metric scale, avoiding auxiliary reconstruction targets that increase computation and optimization complexity. To learn such cross-camera generalization at scale and depth, X-lens is trained on multiple public datasets and OmniScene, our newly released large-scale synthetic dataset containing approximately 266K synchronized six-view frames, 1.7M individual images, and 103 indoor and outdoor scenes. Extensive experiments on both real-world and synthetic indoor and outdoor datasets demonstrate superior heterogeneous-camera metric depth accuracy, reducing AbsRel by 25.4\% on OmniScene-Full over the strongest baseline while using 88.9\% fewer parameters, with competitive performance on conventional fisheye-only and pinhole-only settings.
Figures
Reference graph
Works this paper leans on
-
[1]
Rahul Ahuja, Mudit Jain, Bala Murali Manoghar Sai Sudhakar, Venkatraman Narayanan, Pratik Likhar, Varun Ravi Kumar, and Senthil Yogamani. FishRoPE: Projective rotary position embeddings for omnidirectional visual perception.arXiv preprint arXiv:2604.10391, 2026. 6, 7
Pith/arXiv arXiv 2026
-
[2]
Depth anything in 360: Towards scale invariance in the wild.arXiv preprint arXiv:2512.22819, 2025
Hao Ai, Zhi Cao, Meixi Song, Yuxuan Liu, Haodong Li, Dizhe Zhang, Ming-Hsuan Yang, and Lu Qi. Depth anything in 360: Towards scale invariance in the wild.arXiv preprint arXiv:2512.22819, 2025. 2, 3
arXiv 2025
-
[3]
Scenescript: Reconstructing scenes with an autoregressive structured language model
Armen Avetisyan, Christopher Xie, Henry Howard-Jenkins, Tsun-Yi Yang, Samir Aroudj, Suvam Patra, Fuyang Zhang, Duncan Frost, Luke Holland, Campbell Orme, Jakob Engel, Edward Miller, Richard Newcombe, and Vasileios Balntas. Scenescript: Reconstructing scenes with an autoregressive structured language model. InEuropean Conference on Computer Vision (ECCV), 2024. 9
2024
-
[4]
AdaBins: Depth estimation using adaptive bins
Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. AdaBins: Depth estimation using adaptive bins. InCVPR, 2021. 3
2021
-
[5]
Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias M ¨uller. ZoeDepth: Zero-shot transfer by combining relative and metric depth.arXiv preprint arXiv:2302.12288, 2023. 2, 3
Pith/arXiv arXiv 2023
-
[6]
Aleksei Bochkovskii, Amael Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R. Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second.arXiv preprint arXiv:2410.02073, 2024. 3
Pith/arXiv arXiv 2024
-
[7]
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alex Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuh...
Pith/arXiv arXiv 2023
-
[8]
Jake Bruce, Michael D. Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Bechtle, Feryal Behbahani, Stephanie C. Y . Chan, Nicolas Heess, Laura Gonzalez, Simon Osindero, Sherjil Ozair, Scott Reed, Jingwei Zhang, Konrad Zolna, Jeff Clune, Tim Rocktaschel, S...
Pith/arXiv arXiv 2024
-
[9]
D ´ej`a View: Looping transformers for multi-view 3d reconstruction
Alessandro Burzio, Tobias Fischer, Sven Elflein, Qunjie Zhou, Riccardo de Lutio, Jiawei Ren, Jiahui Huang, Shengyu Huang, Marc Pollefeys, Laura Leal-Taix´e, Zan Gojcic, and Haithem Turki. D ´ej`a View: Looping transformers for multi-view 3d reconstruction. arXiv preprint arXiv:2605.30215, 2026. 4
Pith/arXiv arXiv 2026
-
[10]
Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation.ArXiv, 2025
Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Qiwei Liang, Zixuan Li, Xianlian Lin, Yiheng Ge, Zhenyu Gu, Weiliang Deng, Yubin Guo, Tian Nian, Xuanbing Xie, Qiangyu Chen, Kailun Su, Tianling Xu, Guodong Liu, Mengkang Hu, Huan ang Gao, Kaixuan Wang, Zhixuan Liang, Yusen Qin, Xiaokang Yang, Ping Luo, and Yao Mu. Robotwin 2.0: A scalable da...
2025
-
[11]
Youyu Chen, Junjun Jiang, Yueru Luo, Kui Jiang, Xianming Liu, Xu Yan, and Dave Zhenyu Chen. Reliev3R: Relieving feed-forward reconstruction from multi-view geometric annotations.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21860–21869, 2026. 4
2026
-
[12]
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. InICLR, 2021. 6
2021
-
[13]
Depth map prediction from a single image using a multi-scale deep network
David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep network. In Advances in Neural Information Processing Systems, 2014. 3
2014
-
[14]
Adarsh Ganesan, Sravan Puligilla, Ruicheng Zhang, Kamal Joshi, Sanja Fidler, and David F. Fouhey. UniDAC: Universal metric depth estimation for any camera. InCVPR, 2026. 3, 10, 11, 13, 14, S3
2026
-
[15]
Extending foundational monocular depth estimators to fisheye cameras with calibration tokens
Rit Gangopadhyay, Jung-Hee Kim, Xien Chen, Patrick Rim, Hyoungseob Park, and Alex Wong. Extending foundational monocular depth estimators to fisheye cameras with calibration tokens. InICCV, 2025. 6
2025
-
[16]
Cl ´ement Godard, Oisin Mac Aodha, Michael Firman, and Gabriel J. Brostow. Digging into self-supervised monocular depth estima- tion. InICCV, 2019. 3
2019
-
[17]
3D packing for self-supervised monocular depth estimation
Vitor Guizilini, Rares Ambrus, Sudeep Pillai, Allan Raventos, and Adrien Gaidon. 3D packing for self-supervised monocular depth estimation. InCVPR, 2020. 3
2020
-
[18]
Jing He, Haodong Li, Sili Chen, Jingkai Wang, Zhenyu Li, Xiaokang Chen, and Lei Zhang. Lotus: Diffusion-based visual foundation model for high-quality dense prediction.arXiv preprint arXiv:2409.18124, 2024. 3
Pith/arXiv arXiv 2024
-
[19]
Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, and Chunhua Shen. Metric3D v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation.arXiv preprint arXiv:2404.15506, 2024. 3, 10, 11
Pith/arXiv arXiv 2024
-
[20]
Yeh, and Alexander G
Yuan-Ting Hu, Jiahong Wang, Raymond A. Yeh, and Alexander G. Schwing. SAIL-VOS 3D: A synthetic dataset and baselines for object detection and 3d mesh reconstruction from video data. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 9
2021
-
[21]
Deepmvs: Learning multi-view stereopsis
Po-Han Huang, Kevin Matzen, Johannes Kopf, Narendra Ahuja, and Jia-Bin Huang. Deepmvs: Learning multi-view stereopsis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 9
2018
-
[22]
MVSAnywhere: Zero-shot multi-view stereo.arXiv preprint arXiv:2503.22430,
Sergio Izquierdo, Mohamed Sayed, Michael Firman, Guillermo Garcia-Hernando, Daniyar Turmukhambetov, Javier Civera, Oisin Mac Aodha, Gabriel Brostow, and Jamie Watson. MVSAnywhere: Zero-shot multi-view stereo.arXiv preprint arXiv:2503.22430,
-
[23]
UniFuse: Unidirectional fusion for 360 panorama depth estimation
Hualie Jiang, Zheng Sheng, Siyu Zhu, Zilong Dong, and Rui Huang. UniFuse: Unidirectional fusion for 360 panorama depth estimation. InICRA, 2021. 2, 3
2021
-
[24]
Lihan Jiang, Yucheng Mao, Linning Xu, Tao Lu, Kerui Ren, Yichen Jin, Xudong Xu, Mulin Yu, Jiangmiao Pang, Feng Zhao, et al. AnySplat: Feed-forward 3d gaussian splatting from unconstrained views.arXiv preprint arXiv:2505.23716, 2025. 4
arXiv 2025
-
[25]
Barron, Noah Snavely, and Aleksander Holynski
Haian Jin, Rundi Wu, Tianyuan Zhang, Ruiqi Gao, Jonathan T. Barron, Noah Snavely, and Aleksander Holynski. ZipMap: Linear- time stateful 3d reconstruction with test-time training.arXiv preprint arXiv:2603.04385, 2026. 4
Pith/arXiv arXiv 2026
-
[26]
Dongki Jung, Jaehoon Choi, Adil Qureshi, Somi Jeong, Dinesh Manocha, and Suyong Yeon. Wid3R: Wide field-of-view 3d recon- struction via camera model conditioning.arXiv preprint arXiv:2602.05321, 2026. 4
arXiv 2026
-
[27]
Dynamicstereo: Consistent dynamic depth from stereo videos.CVPR, 2023
Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Dynamicstereo: Consistent dynamic depth from stereo videos.CVPR, 2023. 9
2023
-
[28]
MapAnything: Universal feed-forward metric 3d reconstruction.arXiv preprint arXiv:2509.13414, 2025
Nikhil Keetha, Norman M ¨uller, Johannes Sch ¨onberger, Lorenzo Porzi, Yuchen Zhang, Tobias Fischer, Arno Knapitsch, Duncan Zauss, Ethan Weber, Nelson Antunes, Jonathon Luiten, Manuel Lopez-Antequera, Samuel Rota Bul `o, Christian Richardt, Deva Ramanan, Sebastian Scherer, and Peter Kontschieder. MapAnything: Universal feed-forward metric 3d reconstructio...
Pith/arXiv arXiv 2025
-
[29]
OpenVLA: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Chelsea Finn, and Percy Liang. OpenVLA: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024. 2
Pith/arXiv arXiv 2024
-
[30]
From big to small: Multi-scale local planar guidance for monocular depth estimation
Jin Han Lee, Myung-Kyu Han, Dong Wook Ko, and Il Hong Suh. From big to small: Multi-scale local planar guidance for monocular depth estimation. InarXiv preprint arXiv:1907.10326, 2019. 3
Pith/arXiv arXiv 1907
-
[31]
Grounding image matching in 3d with MASt3R.arXiv preprint arXiv:2406.09756, 2024
Vincent Leroy, Yohann Cabon, and J ´erˆome Revaud. Grounding image matching in 3d with MASt3R.arXiv preprint arXiv:2406.09756, 2024. 2, 4
Pith/arXiv arXiv 2024
-
[32]
Yuyan Li, Sheng Gu, Christoph Mayer, Luc Van Gool, and Radu Timofte. Fisheyedistancenet: Self-supervised scale-aware distance estimation using monocular fisheye camera for autonomous driving.arXiv preprint arXiv:2112.13842, 2021. 3
Pith/arXiv arXiv 2021
-
[33]
CasOmniMVS: Cascade omnidirectional depth estimation with dynamic spherical sweeping.Applied Sciences, 2024
Yuyan Li, Peng Wang, Lingjie Liu, and Wenping Wang. CasOmniMVS: Cascade omnidirectional depth estimation with dynamic spherical sweeping.Applied Sciences, 2024. 3
2024
-
[34]
Megadepth: Learning single-view depth prediction from internet photos
Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 9
2018
-
[35]
Zhenyu Li, Xuyang Wang, Bingyi Kang, Jiashi Feng, and Hengshuang Zhao. PatchRefiner v2: Fast and lightweight real-domain high-resolution metric depth estimation.arXiv preprint arXiv:2501.01121, 2025. 3
Pith/arXiv arXiv 2025
-
[36]
Zhenyu Li, Xuyang Wang, Bingyi Kang, Jiashi Feng, and Hengshuang Zhao. Language as prior, vision as calibration: Metric scale recovery for monocular depth estimation.arXiv preprint arXiv:2601.01457, 2026. 3
arXiv 2026
-
[37]
KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d
Yiyi Liao, Jun Xie, and Andreas Geiger. KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3292–3310, 2023. 9, 10, 11, S3
2023
-
[38]
Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang
Haotong Lin, Sili Chen, Junhao Liew, Donny Y . Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang. Depth anything 3: Recovering the visual space from any views.arXiv preprint arXiv:2511.10647, 2025. 4, 8, 10, 12, 14
Pith/arXiv arXiv 2025
-
[39]
Siyou Lin, Zhou Xue, Hongwen Zhang, Liang An, Dongping Li, Shaohui Jiao, and Yebin Liu. Mix3R: Mixing feed-forward reconstruction and generative 3d priors for joint multi-view aligned 3d reconstruction and pose estimation.arXiv preprint arXiv:2605.03359, 2026. 4
Pith/arXiv arXiv 2026
-
[40]
Depth any panoramas: A foundation model for panoramic depth estimation
Xin Lin, Meixi Song, Dizhe Zhang, Wenxuan Lu, Haodong Li, Bo Du, Ming-Hsuan Yang, Truong Nguyen, and Lu Qi. Depth any panoramas: A foundation model for panoramic depth estimation. InCVPR, 2026. 2, 3, 4
2026
-
[41]
DL3DV-10K: A large-scale scene dataset for deep learning-based 3d vision
Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, Xuanmao Li, Xingpeng Sun, Rohan Ashok, Aniruddha Mukherjee, Hao Kang, Xiangrui Kong, Gang Hua, Tianyi Zhang, Bedrich Benes, and Aniket Bera. DL3DV-10K: A large-scale scene dataset for deep learning-based 3d vision. InProceedings of the IEEE/CVF Conf...
2024
-
[42]
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In International Conference on Learning Representations (ICLR), 2023. S1
2023
-
[43]
Mapillary planet-scale depth dataset
Manuel L ´opez Antequera, Pau Gargallo, Markus Hofinger, Samuel Rota Bul `o, Yubin Kuang, and Peter Kontschieder. Mapillary planet-scale depth dataset. InEuropean Conference on Computer Vision (ECCV), 2020. 9
2020
-
[44]
Wenxuan Lu, Xin Lin, Meixi Song, Dizhe Zhang, Haodong Li, and Lu Qi. WideDepth: Millimeter-accurate benchmark for fisheye depth estimation.arXiv preprint arXiv:2605.24074, 2026. 3
Pith/arXiv arXiv 2026
-
[45]
SimpleProc: Fully procedural synthetic data from simple rules for multi-view stereo
Zeyu Ma, Alexander Raistrick, and Jia Deng. SimpleProc: Fully procedural synthetic data from simple rules for multi-view stereo. arXiv preprint arXiv:2604.04925, 2026. 4
Pith/arXiv arXiv 2026
-
[46]
Spring: A high-resolution high-detail dataset and benchmark for scene flow, optical flow and stereo
Lukas Mehl, Jenny Schmalfuss, Azin Jahedi, Yaroslava Nalivayko, and Andr ´es Bruhn. Spring: A high-resolution high-detail dataset and benchmark for scene flow, optical flow and stereo. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4981–4991, 2023. 9
2023
-
[47]
Andreas Meuleman, HyunJun Jang, Dongju Kim, Dae Gyu Jeon, and Min H. Kim. Real-time sphere sweeping stereo from multiview fisheye images. InCVPR, 2021. 3
2021
-
[48]
V o-dp: Semantic-geometric adaptive diffusion policy for vision-only robotic manipulation.ArXiv, 2025
Zehao Ni, Yonghao He, Lingfeng Qian, Jilei Mao, Fa Fu, Wei Sui, Hu Su, Junran Peng, Zhipeng Wang, and Bin He. V o-dp: Semantic-geometric adaptive diffusion policy for vision-only robotic manipulation.ArXiv, 2025. S1
2025
-
[49]
Octo: An open-source generalist robot policy.arXiv preprint arXiv:2405.12213, 2024
Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Charles Xu, Jianlan Luo, Tess Kreiman, You Liang Tan, Dorsa Chen, Pannag Sanketi, Quan Vuong, Ted Xiao, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Octo: An open-source generalist robot policy.arXiv preprint arXiv:2405.12213, 2024. 2
Pith/arXiv arXiv 2024
-
[50]
DINOv2: Learning robust visual features without supervision.Transactions on Machine Learning Research (TMLR), 2024
Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. DINOv2: Learning robust visual features without supervision.Transactions on Machine Learning Research (TMLR), 2024. 2
2024
-
[51]
TartanGround: A large-scale dataset for ground robot perception and navigation
Manthan Patel, Fan Yang, Yuheng Qiu, Cesar Cadena, Sebastian Scherer, Marco Hutter, and Wenshan Wang. TartanGround: A large-scale dataset for ground robot perception and navigation. In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025. 9
2025
-
[52]
Scalable diffusion models with transformers
William Peebles and Saining Xie. Scalable diffusion models with transformers. InIEEE/CVF International Conference on Computer Vision (ICCV), 2023. S1
2023
-
[53]
UniDepth: Universal monocular metric depth estimation
Luigi Piccinelli, Christos Sakaridis, Yung-Hsu Yang, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. UniDepth: Universal monocular metric depth estimation. InCVPR, 2024. 2, 3
2024
-
[54]
Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. UniDepthV2: Universal monocular metric depth estimation made simpler.arXiv preprint arXiv:2502.20110, 2025. 3, 10, 11
Pith/arXiv arXiv 2025
-
[55]
UniK3D: Universal camera monocular 3d estimation.arXiv preprint arXiv:2503.16591, 2025
Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. UniK3D: Universal camera monocular 3d estimation.arXiv preprint arXiv:2503.16591, 2025. 3
Pith/arXiv arXiv 2025
-
[56]
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InInternational Conference on Machine Learning (ICML), 2021. S1
2021
-
[57]
Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
Ren ´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. 3
2020
-
[58]
Vision transformers for dense prediction
Ren ´e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. InICCV, 2021. 2, 3
2021
-
[59]
Sch¨onberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger
Thomas Sch ¨ops, Johannes L. Sch¨onberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger. A multi-view stereo benchmark with high-resolution images and multi-camera videos. InCVPR, 2017. 10, 11, 12, S3
2017
-
[60]
PanoFormer: Panorama transformer for indoor 360 depth estimation
Zhijie Shen, Chunyu Lin, Kang Liao, Lang Nie, Zishuo Zheng, and Yao Zhao. PanoFormer: Panorama transformer for indoor 360 depth estimation. InECCV, 2022. 2, 3
2022
-
[61]
Oriane Sim’eoni, Huy V . V o, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michael Ramamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timoth ´ee Darcet, Th ´eo Moutakanni, Leonel Sentana, Claire Roberts, Andrea Vedaldi, Jamie Tolan, John Brandt, Camille Couprie, ...
2025
-
[62]
RoFormer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Bo Wen, and Yunfeng Liu. RoFormer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024
2024
-
[63]
HoHoNet: 360 indoor holistic understanding with latent horizontal features
Cheng Sun, Ching-Yu Hsiao, Min Sun, and Hwann-Tzong Chen. HoHoNet: 360 indoor holistic understanding with latent horizontal features. InCVPR, 2021. 3
2021
-
[64]
Distortion-aware convolutional filters for dense prediction in panoramic images
Keisuke Tateno, Nassir Navab, and Federico Tombari. Distortion-aware convolutional filters for dense prediction in panoramic images. InECCV, 2018. 3
2018
-
[65]
Smd-nets: Stereo mixture density networks
Fabio Tosi, Yiyi Liao, Carolin Schmitt, and Andreas Geiger. Smd-nets: Stereo mixture density networks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 9
2021
-
[66]
Generative camera dolly: Extreme monocular dynamic novel view synthesis
Basile Van Hoorick, Rundi Wu, Ege Ozguroglu, Kyle Sargent, Ruoshi Liu, Pavel Tokmakov, Achal Dave, Changxi Zheng, and Carl V ondrick. Generative camera dolly: Extreme monocular dynamic novel view synthesis. InEuropean Conference on Computer Vision (ECCV), 2024. 9
2024
-
[67]
BiFuse: Monocular 360 depth estimation via bi- projection fusion
Fu-En Wang, Yu-Hsuan Yeh, Min Sun, Wei-Chen Chiu, and Yi-Hsuan Tsai. BiFuse: Monocular 360 depth estimation via bi- projection fusion. InCVPR, 2020. 3
2020
-
[68]
V oyager: An open-ended embodied agent with large language models
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. V oyager: An open-ended embodied agent with large language models. InNeurIPS, 2023. 2
2023
-
[69]
3d reconstruction with spatial memory.arXiv preprint arXiv:2408.16061, 2024
Hengyi Wang and Lourdes Agapito. 3d reconstruction with spatial memory.arXiv preprint arXiv:2408.16061, 2024. 4
Pith/arXiv arXiv 2024
-
[70]
VGGT: Visual geometry grounded transformer.arXiv preprint arXiv:2503.11651, 2025
Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. VGGT: Visual geometry grounded transformer.arXiv preprint arXiv:2503.11651, 2025. 2, 4, 5, 10, 11, 12, S1
Pith/arXiv arXiv 2025
-
[71]
VGGT-Ω: Scaling feed-forward 3d reconstruction.arXiv preprint arXiv:2605.15195, 2026
Jianyuan Wang, Minghao Chen, Shangzhan Zhang, Nikita Karaev, Johannes Sch ¨onberger, Patrick Labatut, Piotr Bojanowski, David Novotny, Andrea Vedaldi, and Christian Rupprecht. VGGT-Ω: Scaling feed-forward 3d reconstruction.arXiv preprint arXiv:2605.15195, 2026. 2, 4, 10, 12
Pith/arXiv arXiv 2026
-
[72]
Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. OmniMVS: End-to-end learning for omnidirectional stereo matching.arXiv preprint arXiv:2203.10759, 2022. 3
Pith/arXiv arXiv 2022
-
[73]
Peng Wang, Yuan Liu, Lingjie Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Unsupervised OmniMVS: Efficient omnidirectional depth inference via establishing pseudo-stereo supervision.arXiv preprint arXiv:2302.09922, 2023. 3
Pith/arXiv arXiv 2023
-
[74]
Ruicheng Wang, Sicheng Chen, Jiayuan Xu, Xingyi Li, Zhengqi Li, Zhizheng Zhang, Juefei-Xu Liu, and Ziwei Liu. MoGe: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision.arXiv preprint arXiv:2410.19115, 2024. 3, 8, 14
Pith/arXiv arXiv 2024
-
[75]
Ruicheng Wang, Sicheng Chen, Jiayuan Xu, Xingyi Li, Zhengqi Li, Zhizheng Zhang, Juefei-Xu Liu, and Ziwei Liu. MoGe-2: Accurate monocular geometry with metric scale and sharp details.arXiv preprint arXiv:2507.02546, 2025. 3, 8
Pith/arXiv arXiv 2025
-
[76]
Ruicheng Wang, Sicheng Chen, Jiayuan Xu, Xingyi Li, Zhengqi Li, Zhizheng Zhang, Juefei-Xu Liu, and Ziwei Liu. AnchorD: Metric grounding of monocular depth using factor graphs.arXiv preprint arXiv:2605.02667, 2026. 3
Pith/arXiv arXiv 2026
-
[77]
Ruicheng Wang, Sicheng Chen, Jiayuan Xu, Xingyi Li, Zhengqi Li, Zhizheng Zhang, Juefei-Xu Liu, and Ziwei Liu. MetricAnything: Scaling metric depth pretraining with noisy heterogeneous sources.arXiv preprint arXiv:2601.22054, 2026. 3
Pith/arXiv arXiv 2026
-
[78]
DUSt3R: Geometric 3d vision made easy
Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. DUSt3R: Geometric 3d vision made easy. arXiv preprint arXiv:2312.14132, 2023. 2, 4, 5
Pith/arXiv arXiv 2023
-
[79]
Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, and Tong He.π 3: Scalable permutation-equivariant visual geometry learning.arXiv preprint arXiv:2507.13347, 2025. 2, 4
Pith/arXiv arXiv 2025
-
[80]
Omnimvs: End-to-end learning for omnidirectional stereo matching
Changhee Won, Francois Rameau, Jinwoo Kim, and In So Kweon. Omnimvs: End-to-end learning for omnidirectional stereo matching. InICCV, 2019. 3
2019
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.