Pith. sign in

REVIEW 3 major objections 4 minor 79 references

Towards In-the-wild 3D Plane Reconstruction from a Single Image

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read Single-image 3D plane reconstruction can generalize across indoor and outdoor scenes if trained on a large mixed-domain benchmark with geometry-disentangled, exemplar-guided estimation.

desk verdict A solid, well-executed first shot at zero-shot cross-domain plane reconstruction; the NYUv2 evidence holds up, but the in-the-wild claim is partly undermined by evaluation sets sharing the training label pipeline. read the letter →

arxiv 2506.02493 v1 pith:GVVF5KBD submitted 2025-06-03 cs.CV

classification cs.CV
keywords 3Dplanereconstructionzero-shotgeneralizationsingle-imageestimationTransformerdetectionnormalandoffsetmixed-domaintrainingpanopticsegmentationannotationmonocularsceneunderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that single-image 3D plane reconstruction can be made zero-shot: a model trained on mixed indoor and outdoor data can recover planes from unseen scenes without fine-tuning. To enable that, the authors assemble a benchmark of over 560,000 high-resolution training samples from 14 diverse datasets, including synthetic and real, indoor and street environments, and they generate dense plane labels for datasets that lacked them. The model, named ZeroPlane, treats planes as learnable queries in a Transformer decoder, but changes how plane geometry is learned: normal and offset are disentangled, each mapped to a cluster 'exemplar' first and then refined by a small regression residual, and plane queries are allowed to attend to pixel-level depth and normal predictions. The experiments report consistent gains over prior Transformer baselines on four unseen evaluation sets, with the largest improvements on outdoor and in-the-wild data. If the claim holds, one network can replace per-domain plane models for augmented reality, navigation, and mapping.

What carries the argument

The machinery that carries the argument is the disentangled, exemplar-guided classification-then-regression head for plane geometry, together with a pixel-geometry-enhanced plane embedding module. For each detected plane, two classification heads choose the closest normal exemplar $\hat{n}^{(i)}$ and offset exemplar $\hat{d}^{(j)}$ from clusters precomputed by K-Means on the training set, and two regression heads predict residuals so that $n = \hat{n}^{(i)} + r_n^{(i)}$ and $d = \hat{d}^{(j)} + r_d^{(j)}$. This turns a difficult continuous regression into a discrete choice plus a small correction. In parallel, the decoder computes per-pixel depth and normal maps as auxiliary outputs, projects them into embedding space, and lets the plane queries attend to those projected maps, adding the attended geometric features to the original image-feature attention. That design is what keeps plane geometry accurate when training data spans very different geometric scales.

What would settle it

Take a random sample of 7-Scenes images, re-annotate them with human plane fitting on the back-projected RGB-D point clouds without using the panoptic segmentation model, and rerun ZeroPlane and the baseline on that re-annotated set. If the 0.1m depth-recall and 5-degree normal-recall gaps against the baseline shrink to the levels seen on NYUv2, where labels came from a different source, that would confirm that the shared annotation pipeline was inflating the zero-shot result.

Watch

Extended reading notes

Core claim

On the paper's own terms, ZeroPlane is presented as the first unified, transferable framework for cross-domain 3D plane reconstruction from a single image. The central claim is that broadening the training distribution is not enough: representing a plane as a single scaled vector n/d and regressing it directly fails when indoor and outdoor scenes are mixed because the geometric scales and orientations are too diverse. ZeroPlane instead decouples the plane into a normal and an offset, learns a small set of normal exemplars and offset exemplars by clustering the training planes, classifies each detected plane against those exemplars, and then regresses a residual to the chosen exemplar. It also feeds pixel-level depth and normal predictions back into the plane queries through attention, so the queries carry low-level geometric cues into the final mask and parameter heads. The paper reports that this design, trained on the mixed benchmark, raises zero-shot plane recall on NYUv2 at the 0.1m depth threshold from 14.29 to 17.86 and at the 5-degree normal threshold from 24.97 to 37.29 relative to the prior Transformer baseline trained under the same mixed setting, with similar or larger gains on a second indoor set and on two outdoor sets.

Load-bearing premise

The weakest load-bearing assumption is the validity of the 7-Scenes zero-shot evaluation set: its ground-truth planes were produced by the same automatic panoptic-segmentation-plus-RANSAC pipeline used to build training labels, so part of the reported advantage could come from sharing that pipeline rather than from disentangled plane geometry or geometry-enhanced embeddings.

Editorial extensions

If this is right

  • A single model can serve new environments — unseen rooms, street views, and synthetic scenes — without fine-tuning, which is the precondition for portable augmented-reality plane placement and robot navigation.
  • The roughly 560K-sample mixed benchmark gives the community a common training and evaluation ground for cross-domain plane reconstruction, so future methods can be compared on the same indoor-and-outdoor mix.
  • Disentangling normal from offset and classifying before regressing appears to be the key to stable multi-dataset geometry learning, and the same recipe may transfer to other continuous geometric outputs such as surface normals or depths.
  • High-resolution annotations (640x480) improve both segmentation and geometry, so resolution should be treated as a first-class axis when building plane-reconstruction datasets and models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A confound the paper leaves open: the 7-Scenes evaluation labels were produced by the same panoptic-segmentation-plus-RANSAC pipeline used for several training labels, so the magnitude of the reported zero-shot gain could shrink under a differently labeled evaluation.
  • If a real outdoor metric-depth set were added to training, the outdoor recall numbers would directly test whether the current bottleneck is data domain or representation design; the paper itself cites the lack of such data as a limitation.
  • Because the pixel-geometry branch proved robust to noisy depth and normal inputs, a pretrained depth or normal estimator could plausibly substitute for the auxiliary heads, giving similar plane quality at lower training cost.
  • Because the ablations show exemplar-count choices barely change results, an adaptive version that re-clusters exemplars per test image or per test domain is a plausible cheap upgrade.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper presents ZeroPlane, a Transformer-based single-image 3D plane reconstruction framework designed for zero-shot generalization across indoor and outdoor domains. The authors introduce a large-scale benchmark spanning 14 datasets, with new plane labels generated automatically by a Mask2Former panoptic-segmentation plus RANSAC plane-fitting pipeline. The method decouples plane normal and offset, uses a classification-then-regression scheme with K-means exemplars, integrates a DINOv2/DPT backbone/decoder, and adds a pixel-geometry-enhanced plane-embedding module with auxiliary depth and normal prediction. Experiments on four zero-shot datasets (NYUv2, 7-Scenes, ParallelDomain, ApolloScape) report substantial gains over the PlaneRecTR baseline, especially in normal and strict depth recall. Code and data are promised for release.

Significance. The work is potentially significant: it is the first systematic attempt at cross-domain zero-shot 3D plane reconstruction, the dataset of over 560,000 plane annotations is a substantial community resource, and the proposed components—disentangled classification-then-regression and geometric cross-attention—are clearly motivated and ablated. On NYUv2, whose labels are independent of the training-label pipeline, ZeroPlane outperforms PlaneRecTR by a wide margin (e.g., depth recall @0.1m 17.86 vs 14.29, normal recall @5° 37.29 vs 24.97), and the supplementary ablation that removes Mask2Former-produced training data still shows gains on NYUv2. However, the headline claim 'especially over in-the-wild data' rests partly on evaluation sets (7-Scenes, ParallelDomain, ApolloScape) whose ground truth comes from the same Mask2Former+RANSAC pipeline used to generate training labels. The stress-test concern therefore lands: the measured improvements on those three sets may partly reflect label-pipeline alignment rather than genuinely better geometry. This is a fixable issue, but it is central to the paper's main claim.

major comments (3)
  1. [Sec. 3.2 / Table 2 / Supp A] The zero-shot evaluation sets 7-Scenes, ParallelDomain, and ApolloScape use ground-truth plane labels generated by the same Mask2Former panoptic-segmentation plus RANSAC pipeline that produced training labels for DIODE, Taskonomy, Synthia, Virtual KITTI, and Sanpo. The Mask2Former-bias ablation in Supplementary Table 9 removes Mask2Former-involved training data but evaluates only on NYUv2, which has independent PlaneAE annotations. Consequently, the large gains in Table 2 on 7-Scenes, ParallelDomain, and ApolloScape could partly reflect improved alignment with the Mask2Former label-generation process rather than better geometric reconstruction. The paper should either (i) re-evaluate on these three sets with independent human-made or alternate-pipeline labels on at least a subset, (ii) demonstrate that the improvements persist under a different evaluation-label generation pipeline, or (iii) explicitly temper the 'especially over in-the-wild data' claim to the independent-label evidence. As written, the central generalization claim is not fully supported on three of the four zero-shot benchmarks.
  2. [Supp A, User evaluation] Validation of the newly generated plane labels is limited to a subjective user study (84% 'good', 15% 'borderline', 1% 'bad'), which does not quantify geometric fidelity. Since this label-generation pipeline defines both the training signal and most of the evaluation signal, the paper should provide a quantitative label-quality assessment: for example, compare fitted planes against manually annotated planes on a random subset, report plane-parameter errors and mask IoUs, or compare against an independent mesh-based annotation. Without such a check, the benchmark's reliability is not established beyond visual inspection.
  3. [Table 2 / Table 3 / Sec. 5.3] All quantitative results are reported from a single run without error bars, confidence intervals, or significance tests. Some headline comparisons are small in absolute terms (e.g., ParallelDomain @10m: Ours-DINO-B 63.45 vs Ours-DINO-L 64.0; ApolloScape @1m: Ours-DINO-B 10.28 vs PlaneRecTR (M) 10.09). The abstract and Section 5 claim the method 'significantly outperforms' baselines; without variance estimates over multiple seeds or paired significance tests, readers cannot judge the statistical reliability of these differences. Please report mean and standard deviation over at least three seeds for the main zero-shot table and the key ablations, or clearly state which differences are within expected noise.
minor comments (4)
  1. [Sec. 5.3] The word 'versability' should be 'versatility'.
  2. [References, [25]] The author list for reference [25] contains a malformed entry ('Rare s, Ambrus'); please correct the citation metadata.
  3. [Table 2 caption] The baseline PlaneRCNN is listed as trained on ScanNetv2 (S-v2) while the other ScanNet-trained baselines use ScanNetv1 (S); this asymmetry should be stated in the caption or in the text to avoid confusion about the comparison.
  4. [Figure 2] The caption of Figure 2 is very long and describes the entire pipeline; consider moving part of this description into the main text or supplementary material.

Circularity Check

1 steps flagged · score 2.0 of 10

Zero-shot generalization claim is partially self-referential: three of four evaluation benchmarks use Mask2Former+RANSAC ground truth generated by the same pipeline that produced training labels, while the model architecture is Mask2Former-based.

  1. other [Sec. 3.2 (Plane Label Generation), Table 1 (zero-shot evaluation rows), Supp. C 'The bias introduced by Mask2Former...'.]
    "For datasets lacking complete semantic labels such as DIODE, Taskonomy, 7-Scenes, Synthia, Virtual KITTI, ApolloScape, and ParallelDomain, we employ a SOTA image segmentation network Mask2Former [14] to obtain their panoptic segmentation results as pseudo ground truth. ... One potential concern raised from our proposed plane annotation pipeline and our framework is that, we use Mask2former's panoptic segmentation predictions for instance segmentation then plane fitting during groundtruth generation for a couple of datasets, while our framework is also partially based on Mask2former."

    The zero-shot evaluation sets 7-Scenes, ParallelDomain, and ApolloScape have ground-truth planes produced by the same Mask2Former panoptic segmentation plus RANSAC fitting pipeline (Sec. 3.2) that generates training labels for DIODE, Taskonomy, Synthia, Virtual KITTI, and Sanpo. The model itself is 'partially based on Mask2former' (the paper's own statement), so high recall on these benchmarks can be achieved by reproducing the labeling pipeline's masks and plane proposals, rather than by reconstructing independently verified plane geometry. The supplementary ablation (Table 9) removes Mask2Former-produced datasets from training but only re-evaluates NYUv2, which has independent annotations; it does not re-test 7-Scenes, ParallelDomain, or ApolloScape with independent labels.

full rationale

The paper is an empirical system paper rather than a formal derivation, and most of its components are not circular: the normal/offset exemplars are clustered from training data, the classification-then-regression prediction depends on image features, and the Mask2Former training-bias is explicitly ablated in Table 9. No load-bearing self-citation chain is present; the only self-citation (PlanemVS [39]) is incidental. The one genuine circularity concern is evaluation-sided: 7-Scenes, ParallelDomain, and ApolloScape are annotated with the same Mask2Former+RANSAC pipeline used to create a substantial portion of the training labels, and the architecture is built on Mask2Former. The paper acknowledges this bias in both training and evaluation, but its ablation only controls training bias and only on NYUv2. Thus the 'especially over in-the-wild data' claim is partially self-referential on those three benchmarks, while the NYUv2 results and the architecture ablations still provide independent evidence. This warrants a low but nonzero circularity score.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the quality and domain coverage of the auto-generated plane labels, the transferability of the pretrained encoder, and the standard camera/plane model. No new physical entities are introduced. The free parameters are mostly cluster counts and the RANSAC fit thresholds, all of which are ablated or shown to be robust.

free parameters (5)
  • Normal exemplar count Kn = 7
    Number of K-Means clusters for plane normals, set by hand; ablation in Table 8 shows low sensitivity.
  • Offset exemplar count Kd = 20 (10 per offset group)
    Clusters for plane offsets, split at the 20m threshold; ablation in Table 8 shows robustness.
  • Offset split threshold = 20m
    Chosen to separate indoor/outdoor scales before clustering; no ablation is provided for this exact threshold.
  • RANSAC distance-aware error constants = 0.05m reference error, 10m reference depth
    Used in supplementary Eq. (4) to adapt the plane fitting error threshold to average depth; hand-picked constants.
  • Loss weights lambda_* = lambda_c=2.0, lambda_m=5.0, lambda_nc=1.0, lambda_nr=5.0, lambda_dc=1.0, lambda_dr=2.0, lambda_pd=0.5…
    Set empirically in Supplementary Sec. B; these influence training balance but are not central to the scientific claim.
assumptions (3)
  • domain assumption The pretrained DINOv2 encoder has not seen the zero-shot evaluation distributions (NYUv2, 7-Scenes, ParallelDomain, ApolloScape), so reported generalization is not due to encoder leakage.
    The authors do not analyze overlap between DINOv2 pretraining data (LVD-142M) and the evaluation datasets; if overlap exists, zero-shot gains could be overstated.
  • domain assumption Mask2Former panoptic segmentation provides a valid candidate set for planar regions in datasets without semantic labels.
    Used in Sec. 3.2 for ground truth generation; quality depends on Mask2Former's generalization to these datasets, and the paper only partially validates this.
  • standard math The pinhole camera model and the standard plane equation n^T X + d = 0 are sufficient for recovering planar depth from predicted plane parameters.
    Utilized in Sec. 4.1 for depth reconstruction; this is standard in the field and not a source of novel error.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards In-the-wild 3D Plane Reconstruction from a Single Image." pith.science (2026). https://pith.science/paper/GVVF5KBD

@misc{pith2026250602493,
  author       = {Pith},
  title        = {Pith review of: Towards In-the-wild 3D Plane Reconstruction from a Single Image},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GVVF5KBD}},
  note         = {Machine review of arXiv:2506.02493}
}
read the original abstract

3D plane reconstruction from a single image is a crucial yet challenging topic in 3D computer vision. Previous state-of-the-art (SOTA) methods have focused on training their system on a single dataset from either indoor or outdoor domain, limiting their generalizability across diverse testing data. In this work, we introduce a novel framework dubbed ZeroPlane, a Transformer-based model targeting zero-shot 3D plane detection and reconstruction from a single image, over diverse domains and environments. To enable data-driven models across multiple domains, we have curated a large-scale planar benchmark, comprising over 14 datasets and 560,000 high-resolution, dense planar annotations for diverse indoor and outdoor scenes. To address the challenge of achieving desirable planar geometry on multi-dataset training, we propose to disentangle the representation of plane normal and offset, and employ an exemplar-guided, classification-then-regression paradigm to learn plane and offset respectively. Additionally, we employ advanced backbones as image encoder, and present an effective pixel-geometry-enhanced plane embedding module to further facilitate planar reconstruction. Extensive experiments across multiple zero-shot evaluation datasets have demonstrated that our approach significantly outperforms previous methods on both reconstruction accuracy and generalizability, especially over in-the-wild data. Our code and data are available at: https://github.com/jcliu0428/ZeroPlane.

Figures

Figures reproduced from arXiv: 2506.02493 by the authors.

Figure 1
Figure 1. Our plane reconstruction framework, ZeroPlane, demon [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our proposed ZeroPlane framework. Taking a single image as input, our model first extracts image features from encoder and decoder networks. The plane queries and the predicted pixel-level depth and normal map serve as inputs to module (a) to obtain geometry-enhanced plane embeddings. These embeddings are then fed into plane detection heads for mask and classification predictions, and plane geometry heads for disent… view at source ↗
Figure 3
Figure 3. Qualitative results from our mix-trained model for zero-shot plane segmentation and mesh reconstruction on NYUv2, 7-Scenes, [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: From top to bottom: our annotated ground-truth planes on HM3D [ [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: From top to bottom: the plane segmentation and reconstruction visualization of our model on ScanNet [ [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

79 extracted references · 61 canonical work pages

  1. [1]

    Planeformers: From sparse view planes to 3d re- construction

    Samir Agarwala, Linyi Jin, Chris Rockwell, and David F Fouhey. Planeformers: From sparse view planes to 3d re- construction. InEuropean Conference on Computer Vision, pages 192–209. Springer, 2022. 2

  2. [2]

    Mapillary planet-scale depth dataset

    Manuel López Antequera, Pau Gargallo, Markus Hofinger, Samuel Rota Bulò, Yubin Kuang, and Peter Kontschieder. Mapillary planet-scale depth dataset. InComputer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, Au- gust 23–28, 2020, Proceedings, Part II 16, pages 589–604. Springer, 2020. 2

  3. [3]

    Mip-nerf 360: Unbounded anti-aliased neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. InProceedings of the IEEE/CVF conference on computer vision and pattern recog- nition, pages 5470–5479, 2022. 1

  4. [4]

    Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data.arXiv preprint arXiv:2111.08897, 2021

    Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, et al. Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data.arXiv preprint arXiv:2111.08897, 2021. 1

  5. [5]

    Adabins: Depth estimation using adaptive bins

    Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4009–4018, 2021. 5

  6. [6]

    Zoedepth: Zero-shot trans- fer by combining relative and metric depth.arXiv preprint arXiv:2302.12288, 2023

    Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias Müller. Zoedepth: Zero-shot trans- fer by combining relative and metric depth.arXiv preprint arXiv:2302.12288, 2023. 2, 3

  7. [7]

    Depth pro: Sharp monocular metric depth in less than a second.arXiv preprint arXiv:2410.02073, 2024

    Aleksei Bochkovskii, Amaël Delaunoy, Hugo Germain, Mar- cel Santos, Yichao Zhou, Stephan R Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second.arXiv preprint arXiv:2410.02073, 2024. 4

  8. [8]

    Vir- tual kitti 2.arXiv preprint arXiv:2001.10773, 2020

    Yohann Cabon, Naila Murray, and Martin Humenberger. Vir- tual kitti 2.arXiv preprint arXiv:2001.10773, 2020. 2, 3, 1

Show all 79 references
  1. [9]

    nuscenes: A multi- modal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. InProceedings of the IEEE/CVF conference on computer vision and patter...

  2. [10]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. InEuropean confer- ence on computer vision, pages 213–229. Springer, 2020. 4, 6

  3. [11]

    Matterport3d: Learning from rgb-d data in indoor environments.arXiv preprint arXiv:1709.06158, 2017

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments.arXiv preprint arXiv:1709.06158, 2017. 2, 3

  4. [12]

    Ninja on a plane: Automatic discovery of physical planes for augmented reality using visual slam

    Denis Chekhlov, Andrew P Gee, Andrew Calway, and Walte- rio Mayol-Cuevas. Ninja on a plane: Automatic discovery of physical planes for augmented reality using visual slam. In 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pages 153–156. IEEE, 2007. 1

  5. [13]

    Oasis: A large-scale dataset for single image 3d in the wild

    Weifeng Chen, Shengyi Qian, David Fan, Noriyuki Kojima, Max Hamilton, and Jia Deng. Oasis: A large-scale dataset for single image 3d in the wild. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 679–688, 2020. 5

  6. [14]

    Masked-attention mask trans- former for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask trans- former for universal image segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022. 2, 4, 6, 1, 3

  7. [15]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. InProceedings of the IEEE conference on computer vision and pattern recog...

  8. [16]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. 1, 2, 3, 7, 5

  9. [17]

    An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint ar...

  10. [18]

    Omnidata: A scalable pipeline for making multi-task mid-level vision datasets from 3d scans

    Ainaz Eftekhar, Alexander Sax, Jitendra Malik, and Amir Zamir. Omnidata: A scalable pipeline for making multi-task mid-level vision datasets from 3d scans. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 10786–10796, 2021. 2, 3

  11. [19]

    Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981

    Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981. 2, 3

  12. [20]

    Deep ordinal regression net- work for monocular depth estimation

    Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Bat- manghelich, and Dacheng Tao. Deep ordinal regression net- work for monocular depth estimation. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2002–2011, 2018. 5

  13. [21]

    Manhattan-world stereo

    Yasutaka Furukawa, Brian Curless, Steven M Seitz, and Richard Szeliski. Manhattan-world stereo. In2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 1422–1429. IEEE, 2009. 1, 2

  14. [22]

    Virtual worlds as proxy for multi-object tracking analysis

    Adrien Gaidon, Qiao Wang, Yohann Cabon, and Eleonora Vig. Virtual worlds as proxy for multi-object tracking analysis. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4340–4349, 2016. 2, 3, 1

  15. [23]

    Piece- wise planar and non-planar stereo for urban scene recon- struction

    David Gallup, Jan-Michael Frahm, and Marc Pollefeys. Piece- wise planar and non-planar stereo for urban scene recon- struction. In2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1418–1425. IEEE, 2010. 1, 2

  16. [24]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In2012 IEEE conference on computer vision and pattern recognition, pages 3354–3361. IEEE, 2012. 1 9

  17. [25]

    Geometric unsupervised domain adaptation for semantic seg- mentation

    Vitor Guizilini, Jie Li, Rare s, Ambrus, , and Adrien Gaidon. Geometric unsupervised domain adaptation for semantic seg- mentation. InProceedings of the IEEE/CVF international conference on computer vision, pages 8537–8547, 2021. 3, 6, 7, 1, 2, 5

  18. [26]

    Towards zero-shot scale-aware monocu- lar depth estimation

    Vitor Guizilini, Igor Vasiljevic, Dian Chen, Rares, Ambrus,, and Adrien Gaidon. Towards zero-shot scale-aware monocu- lar depth estimation. InProceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 9233–9243,

  19. [27]

    Keyframe-based dense planar slam

    Ming Hsiao, Eric Westman, Guofeng Zhang, and Michael Kaess. Keyframe-based dense planar slam. In2017 IEEE In- ternational Conference on Robotics and Automation (ICRA), pages 5110–5117. Ieee, 2017. 1

  20. [28]

    Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation.arXiv preprint arXiv:2404.15506, 2024

    Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation.arXiv preprint arXiv:2404.15506, 2024. 4

  21. [29]

    The apolloscape dataset for autonomous driving

    Xinyu Huang, Xinjing Cheng, Qichuan Geng, Binbin Cao, Dingfu Zhou, Peng Wang, Yuanqing Lin, and Ruigang Yang. The apolloscape dataset for autonomous driving. InProceed- ings of the IEEE conference on computer vision and pattern recognition workshops, pages 954–960, 2018. 2, 3, 6, 7, 1

  22. [30]

    Zeroshape: Regression-based zero-shot shape reconstruction.arXiv preprint arXiv:2312.14198, 2023

    Zixuan Huang, Stefan Stojanov, Anh Thai, Varun Jampani, and James M Rehg. Zeroshape: Regression-based zero-shot shape reconstruction.arXiv preprint arXiv:2312.14198, 2023. 3

  23. [31]

    Planar surface reconstruction from sparse views

    Linyi Jin, Shengyi Qian, Andrew Owens, and David F Fouhey. Planar surface reconstruction from sparse views. InProceed- ings of the IEEE/CVF International Conference on Computer Vision, pages 12991–13000, 2021. 2, 3

  24. [32]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. InProceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 4015–4026, 2023. 2, 8

  25. [33]

    Evaluation of cnn-based single-image depth estima- tion methods

    Tobias Koch, Lukas Liebel, Friedrich Fraundorfer, and Marco Korner. Evaluation of cnn-based single-image depth estima- tion methods. InProceedings of the European Conference on Computer Vision (ECCV) Workshops, pages 0–0, 2018. 1

  26. [34]

    Conditional random fields: Probabilistic models for segment- ing and labeling sequence data

    John Lafferty, Andrew McCallum, and Fernando CN Pereira. Conditional random fields: Probabilistic models for segment- ing and labeling sequence data. 2001. 2

  27. [35]

    Markov random field models in computer vision

    Stan Z Li. Markov random field models in computer vision. InComputer Vision—ECCV’94: Third European Conference on Computer Vision Stockholm, Sweden, May 2–6 1994 Pro- ceedings, Volume II 3, pages 361–370. Springer, 1994. 2

  28. [36]

    Refinenet: Multi-path refinement networks for high- resolution semantic segmentation

    Guosheng Lin, Anton Milan, Chunhua Shen, and Ian Reid. Refinenet: Multi-path refinement networks for high- resolution semantic segmentation. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1925–1934, 2017. 4

  29. [37]

    PlaneNet: Piece-wise planar reconstruction from a single rgb image

    Chen Liu, Jimei Yang, Duygu Ceylan, Ersin Yumer, and Ya- sutaka Furukawa. PlaneNet: Piece-wise planar reconstruction from a single rgb image. InProceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pages 2579–2588, 2018. 1, 2, 3, 6

  30. [38]

    PlaneRCNN: 3d plane detection and reconstruc- tion from a single image

    Chen Liu, Kihwan Kim, Jinwei Gu, Yasutaka Furukawa, and Jan Kautz. PlaneRCNN: 3d plane detection and reconstruc- tion from a single image. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4450–4459, 2019. 1, 2, 3, 5, 7

  31. [39]

    Planemvs: 3d plane reconstruction from multi-view stereo

    Jiachen Liu, Pan Ji, Nitin Bansal, Changjiang Cai, Qin- gan Yan, Xiaolei Huang, and Yi Xu. Planemvs: 3d plane reconstruction from multi-view stereo. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8665–8675, 2022. 1, 2, 5

  32. [40]

    Zero-1-to-3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9298– 9309, 2023. 3

  33. [41]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 4

  34. [42]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017. 6

  35. [43]

    Local light field fusion: Practical view synthe- sis with prescriptive sampling guidelines.ACM Transactions on Graphics (TOG), 38(4):1–14, 2019

    Ben Mildenhall, Pratul P Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, and Abhishek Kar. Local light field fusion: Practical view synthe- sis with prescriptive sampling guidelines.ACM Transactions on Graphics (TOG), 38(4):1–14, 2019. 5

  36. [44]

    Polyfit: Polygonal surface reconstruction from point clouds

    Liangliang Nan and Peter Wonka. Polyfit: Polygonal surface reconstruction from point clouds. InProceedings of the IEEE International Conference on Computer Vision, pages 2353– 2361, 2017. 1

  37. [45]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023. 2, 4, 6

  38. [46]

    Learning pairwise inter-plane relations for piecewise planar reconstruction

    Yiming Qian and Yasutaka Furukawa. Learning pairwise inter-plane relations for piecewise planar reconstruction. In European Conference on Computer Vision, pages 330–345. Springer, 2020. 2, 3, 6, 7, 1, 5

  39. [47]

    René Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer.IEEE transactions on pattern analysis and machine intelligence, 44(3):1623–1637, 2020. 2, 3

  40. [48]

    Vi- sion transformers for dense prediction

    René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction. InProceedings of the IEEE/CVF international conference on computer vision, pages 12179–12188, 2021. 2, 4

  41. [49]

    The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes

    German Ros, Laura Sellart, Joanna Materzynska, David Vazquez, and Antonio M Lopez. The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. InProceedings of the IEEE conference on com- puter vision and pattern recognition, pages 32...

  42. [50]

    Dense planar slam

    Renato F Salas-Moreno, Ben Glocken, Paul HJ Kelly, and Andrew J Davison. Dense planar slam. In2014 IEEE interna- 10 tional symposium on mixed and augmented reality (ISMAR), pages 157–164. IEEE, 2014. 1

  43. [51]

    Zeronvs: Zero-shot 360- degree view synthesis from a single real image.arXiv preprint arXiv:2310.17994, 2023

    Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Herrmann, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry La- gun, Li Fei-Fei, Deqing Sun, et al. Zeronvs: Zero-shot 360- degree view synthesis from a single real image.arXiv preprint arXiv:2310.17994, 2023. 3

  44. [52]

    A multi-view stereo benchmark with high- resolution images and multi-camera videos

    Thomas Schops, Johannes L Schonberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and An- dreas Geiger. A multi-view stereo benchmark with high- resolution images and multi-camera videos. InProceedings of the IEEE conference on computer vision and pat...

  45. [53]

    Planerectr: Unified query learning for 3d plane recovery from a single view

    Jingjia Shi, Shuaifeng Zhi, and Kai Xu. Planerectr: Unified query learning for 3d plane recovery from a single view. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9377–9386, 2023. 1, 2, 4, 5, 6, 7, 8, 3

  46. [54]

    Scene coordinate regression forests for camera relocalization in rgb- d images

    Jamie Shotton, Ben Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, and Andrew Fitzgibbon. Scene coordinate regression forests for camera relocalization in rgb- d images. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2930–29...

  47. [55]

    Indoor segmentation and support inference from rgbd images

    Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. InComputer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7- 13, 2012, Proceedings, Part V 12, pages 746–760. Springer,

  48. [56]

    Piecewise planar stereo for image-based rendering

    Sudipta Sinha, Drew Steedly, and Rick Szeliski. Piecewise planar stereo for image-based rendering. InInternational Conference on Computer Vision, pages 1881–1888, 2009. 1, 2

  49. [57]

    The replica dataset: A digital replica of indoor spaces.arXiv preprint arXiv:1906.05797,

    Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, et al. The replica dataset: A digital replica of indoor spaces.arXiv preprint arXiv:1906.05797,

  50. [58]

    Point-plane slam for hand-held 3d sensors

    Yuichi Taguchi, Yong-Dian Jian, Srikumar Ramalingam, and Chen Feng. Point-plane slam for hand-held 3d sensors. In 2013 IEEE international conference on robotics and automa- tion, pages 5182–5189. IEEE, 2013. 1

  51. [59]

    PlaneTR: Structure-guided transformers for 3d plane recovery

    Bin Tan, Nan Xue, Song Bai, Tianfu Wu, and Gui-Song Xia. PlaneTR: Structure-guided transformers for 3d plane recovery. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4186–4195, 2021. 1, 2, 4, 5, 6, 7

  52. [60]

    Nope-sac: Neural one-plane ransac for sparse-view planar 3d reconstruc- tion.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

    Bin Tan, Nan Xue, Tianfu Wu, and Gui-Song Xia. Nope-sac: Neural one-plane ransac for sparse-view planar 3d reconstruc- tion.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. 2

  53. [61]

    Diode: A dense indoor and outdoor depth dataset.arXiv preprint arXiv:1908.00463, 2019

    Igor Vasiljevic, Nick Kolkin, Shanyi Zhang, Ruotian Luo, Haochen Wang, Falcon Z Dai, Andrea F Daniele, Moham- madreza Mostajabi, Steven Basart, Matthew R Walter, et al. Diode: A dense indoor and outdoor depth dataset.arXiv preprint arXiv:1908.00463, 2019. 3, 1

  54. [62]

    Sanpo: A scene understanding, accessibility, navigation, pathfinding, obstacle avoidance dataset.arXiv preprint arXiv:2309.12172, 2023

    Sagar M Waghmare, Kimberly Wilber, Dave Hawkey, Xuan Yang, Matthew Wilson, Stephanie Debats, Cattalyya Nu- engsigkapian, Astuti Sharma, Lars Pandikow, Huisheng Wang, et al. Sanpo: A scene understanding, accessibility, navigation, pathfinding, obstacle avoidance dataset.arXiv p...

  55. [63]

    Dust3r: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 20697–20709, 2024. 6

  56. [64]

    Airplanes: Accurate plane estimation via 3d-consistent embeddings

    Jamie Watson, Filippo Aleotti, Mohamed Sayed, Zawar Qureshi, Oisin Mac Aodha, Gabriel Brostow, Michael Firman, and Sara Vicente. Airplanes: Accurate plane estimation via 3d-consistent embeddings. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti...

  57. [65]

    Zeroavatar: Zero-shot 3d avatar generation from a single image.arXiv preprint arXiv:2305.16411, 2023

    Zhenzhen Weng, Zeyu Wang, and Serena Yeung. Zeroavatar: Zero-shot 3d avatar generation from a single image.arXiv preprint arXiv:2305.16411, 2023. 3

  58. [66]

    Planarrecon: Real-time 3d plane detection and reconstruction from posed monocular videos

    Yiming Xie, Matheus Gadelha, Fengting Yang, Xiaowei Zhou, and Huaizu Jiang. Planarrecon: Real-time 3d plane detection and reconstruction from posed monocular videos. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6219–6228, 2022. 1, 2, 5

  59. [67]

    Habitat-matterport 3d semantics dataset

    Karmesh Yadav, Ram Ramrakhya, Santhosh Kumar Ramakr- ishnan, Theo Gervet, John Turner, Aaron Gokaslan, Noah Maestre, Angel Xuan Chang, Dhruv Batra, Manolis Savva, et al. Habitat-matterport 3d semantics dataset. InProceedings of the IEEE/CVF Conference on Computer Vision and Pa...

  60. [68]

    Recovering 3d planes from a single image via convolutional neural networks

    Fengting Yang and Zihan Zhou. Recovering 3d planes from a single image via convolutional neural networks. InProceed- ings of the European Conference on Computer Vision, pages 85–100, 2018. 1, 2, 3

  61. [69]

    Depth anything: Unleash- ing the power of large-scale unlabeled data.arXiv preprint arXiv:2401.10891, 2024

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleash- ing the power of large-scale unlabeled data.arXiv preprint arXiv:2401.10891, 2024. 2, 3, 8

  62. [70]

    Depth anything v2.arXiv preprint arXiv:2406.09414, 2024

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2.arXiv preprint arXiv:2406.09414, 2024. 3

  63. [71]

    Polymax: General dense prediction with mask transformer

    Xuan Yang, Liangzhe Yuan, Kimberly Wilber, Astuti Sharma, Xiuye Gu, Siyuan Qiao, Stephanie Debats, Huisheng Wang, Hartwig Adam, Mikhail Sirotenko, et al. Polymax: General dense prediction with mask transformer. InProceedings of the IEEE/CVF Winter Conference on Applications of...

  64. [72]

    Self- supervised super-plane for neural 3d reconstruction

    Botao Ye, Sifei Liu, Xueting Li, and Ming-Hsuan Yang. Self- supervised super-plane for neural 3d reconstruction. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21415–21424, 2023. 2

  65. [73]

    Towards accurate reconstruction of 3d scene shape from a single monocular image.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):6480–6494, 2022

    Wei Yin, Jianming Zhang, Oliver Wang, Simon Niklaus, Si- mon Chen, Yifan Liu, and Chunhua Shen. Towards accurate reconstruction of 3d scene shape from a single monocular image.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):6480–6494, 2022. 2

  66. [74]

    Metric3d: 11 Towards zero-shot metric 3d prediction from a single image

    Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaix- uan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: 11 Towards zero-shot metric 3d prediction from a single image. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9043–9053, 2023. 2, 3

  67. [75]

    Single-image piece-wise planar 3d recon- struction via associative embedding

    Zehao Yu, Jia Zheng, Dongze Lian, Zihan Zhou, and Shenghua Gao. Single-image piece-wise planar 3d recon- struction via associative embedding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1029–1037, 2019. 1, 2, 3, 7

  68. [76]

    Taskonomy: Disentangling task transfer learning

    Amir R Zamir, Alexander Sax, William Shen, Leonidas J Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 3712–3722, 2018. 3, 1, 2

  69. [77]

    3dmatch: Learning local geometric descriptors from rgb-d reconstructions

    Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas Funkhouser. 3dmatch: Learning local geometric descriptors from rgb-d reconstructions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1802–1811, 2017. 3

  70. [78]

    Structural multiplane image: Bridging neural view synthesis and 3d reconstruction

    Mingfang Zhang, Jinglu Wang, Xiao Li, Yifei Huang, Yoichi Sato, and Yan Lu. Structural multiplane image: Bridging neural view synthesis and 3d reconstruction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16707–16716, 2023. 2

  71. [79]

    Scene parsing through ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Bar- riuso, and Antonio Torralba. Scene parsing through ade20k dataset. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 633–641, 2017. 1 12 Towards In-the-wild 3D Plane Reconstructio...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.