Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

ViPOcc: Leveraging Visual Priors from Vision Foundation Models for Single-View 3D Occupancy Prediction

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that frozen vision foundation models can supply the metric scale and instance focus that photometric-only training misses, making single-view 3D occupancy both more accurate and more detailed.

desk verdict Solid incremental system with a portable sampler; the depth SoTA claim rests on the rendered branch rather than the proposed depth branch. read the letter →

arxiv 2412.11210 v2 pith:UTWLRBQP submitted 2024-12-15 cs.CV

Yi Feng , Yu Han , Xijing Zhang , Tanghui Li , Yanting Zhang , Rui Fan This is my paper · ORCID
classification cs.CV
keywords 3Doccupancypredictionsingle-viewreconstructionvisionfoundationmodelsmetricdepthestimationinversealignmentGaussianmixtureraysamplingself-supervisedneuralradiancefieldsautonomousdriving
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that what single-view occupancy prediction has been missing is metric scale and instance awareness, and that frozen vision foundation models can supply both. ViPOcc starts from the metric pseudo depth produced by Depth Anything V2 and learns a residual correction in inverse-depth space, so the recovered depth can hold adjacent frames to the same photometric appearance and the rendered depth to the predicted depth. A semantic-guided non-overlapping Gaussian mixture sampler, built from Grounded-SAM instance masks, concentrates rays on cars and pedestrians while a uniform background term covers roads, buildings, and sky. On KITTI-360 the framework reports scene occupancy accuracy 0.93 versus 0.92 for the BTS and KYN baselines, and on KITTI Raw it reports Abs Rel 0.096 versus 0.102 for BTS. If the claims hold, the result is a training recipe that couples depth and occupancy supervision so each task improves the other without 3D labels.

What carries the argument

The load-bearing identity is the inverse-depth residual $\widehat{D}(x) = 1/(1/D_p(x) + f(F_s, \theta) + \epsilon)$, which converts the VFM's pseudo depth $D_p$ into metric depth $\widehat{D}$ by fitting a correction in inverse-depth space with a convolutional layer on spatial features. Working in inverse depth keeps the correction numerically tame and preserves the local detail of the prior. The second mechanism is the SNOG sampler, whose conditioned PDF $P(x|X)$ is a Gaussian mixture over instance centers plus a background uniform term, with covariance set so about 95.5 percent of samples fall inside each bounding box and with an exclusion radius that forbids overlapping patches. These two mechanisms feed the temporal alignment loss $L_{\mathrm{ta}}$ and the reconstruction consistency losses $L_{\mathrm{rc}}^d$ and $L_{\mathrm{rc}}^{\mathrm{rgb}}$ that couple the depth branch and the NeRF occupancy branch.

What would settle it

A concrete check: evaluate the trained model on a held-out city without median scaling, comparing raw metric predictions to LiDAR; if the inverse-depth residual has learned metric scale, Abs Rel should remain near the KITTI-360 value (about 0.097), whereas if it only fits KITTI's depth distribution the raw error will revert toward the pseudo-depth-with-median-scaling level (about 0.142).

Watch

Extended reading notes

Core claim

The central claim is that a single-view 3D occupancy predictor can be made simultaneously more accurate and more detailed by injecting visual priors from two frozen vision foundation models. Depth Anything V2 supplies metric pseudo depth, but with a large domain gap (0.586 Abs Rel before alignment on KITTI-360); ViPOcc's inverse depth alignment module fits a residual in inverse-depth space and produces refined metric depth that preserves the prior's local detail. That refined depth is then used in a temporal alignment loss between adjacent principal frames and in a reconstruction consistency loss tying rendered depth and RGB patches to the predicted depth. The second prior is semantic: Grounded-SAM instance boxes and masks parameterize a non-overlapping Gaussian mixture sampler, so rays concentrate on crucial objects while background areas are covered uniformly. With these parts coupled in one loss, ViPOcc reports the best occupancy and depth numbers among the compared NeRF-based methods on KITTI-360 and KITTI Raw.

Load-bearing premise

The load-bearing premise is that a residual learned in inverse-depth space, supervised only by photometric and reconstruction consistency, transfers Depth Anything V2's local detail into true metric depth on KITTI-like domains; if that residual instead memorizes the training depth distribution, the metric-depth gains would not survive outside the training distribution.

Editorial extensions

If this is right

  • A single training run produces both metric depth and 3D occupancy, so downstream modules such as planning or mapping could consume either geometry representation without a second training stage.
  • The inverse-depth alignment recipe is defined for whatever residual remains after a domain gap, so it should also absorb depth priors from other metric-depth vision foundation models.
  • The SNOG sampler is reported to be a plug-in improvement: adding it to the BTS and KYN baselines raises their invisible-scene recall, making instance-aware ray allocation a separable contribution.
  • The coupling of temporal photometric alignment with spatial reconstruction consistency is what moves invisible-scene accuracy and recall upward in the ablations, pointing to spatial-temporal consistency as the operative supervisory signal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct testable extension is to swap Depth Anything V2 for a different metric-depth VFM while keeping the same residual module; if the module is genuinely domain-agnostic, gains should track the prior's edge quality rather than its raw metric error.
  • The inverse-depth parameterization is an inductive bias: fitting residuals in reciprocal depth compresses the dynamic range of the VFM's errors, which likely explains why direct depth-residual fitting collapses in the ablations; measuring the error distribution in both spaces would test that explanation.
  • Because the sampler's Gaussian covariance is fixed from 2D bounding boxes, ray density tracks 2D instance size; an alternative allocation weighting rays by expected 3D occupancy gain could perform differently on small distant objects.
  • The supplement notes that zero-shot DDAD evaluation applies median scaling, so true metric generalization outside KITTI is not established; re-running the zero-shot test with the trained scale unnormalized would separate the alignment module's scale recovery from median normalization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes ViPOcc, a single-view 3D occupancy prediction framework that couples a NeRF-based density-field branch with a separate metric depth branch. Depth priors from Depth Anything V2 are aligned to metric depth through an inverse-depth residual module (Eq. 3), and a semantic-guided non-overlapping Gaussian mixture (SNOG) sampler built on Grounded-SAM is used for instance-aware ray sampling. Training combines a temporal alignment loss (Eq. 9), a reconstruction consistency loss between rendered and predicted depth (Eq. 10), and an RGB rendering loss. On KITTI-360 and KITTI Raw, the authors report improved occupancy accuracy over BTS and KYN (0.93 vs. 0.92 scene accuracy) and improved depth metrics (e.g., 0.096 vs. 0.102 Abs Rel on KITTI Raw), plus a zero-shot DDAD result.

Significance. If the central claims are substantiated, the paper makes a useful empirical contribution: it shows that frozen visual priors (Depth Anything V2 and Grounded-SAM) can be integrated into self-supervised NeRF-based occupancy learning, and the proposed SNOG sampler is shown to transfer to BTS and KYN. Strengths of the submission include a clear architecture description, extensive ablations, a released code repository, and auxiliary efficiency measurements. However, the headline depth-estimation claim is currently evaluated on the rendered depth map rather than on the output of the proposed depth branch, and the occupancy gains are small and reported without statistical uncertainty. These gaps need to be addressed before the SoTA claims can be accepted.

major comments (3)
  1. [Supplement B.1.2; Eq. (3); Tables 3, 4, 6] Supplement B.1.2 states that 'we use \hat{D}_r for evaluation and \hat{D} for visualization in all experiments.' This directly affects the central depth-estimation claim: the numbers in Tables 3, 4, and 6 are obtained from the NeRF-rendered depth \hat{D}_r, not from the output \hat{D} of the proposed metric depth branch with the inverse depth alignment module (Eq. 3). No ablation reports metrics for \hat{D}, so the contribution of the inverse depth alignment module to the reported depth gains is untested, and it is possible that the improvement comes from the occupancy/rendering branch rather than from the depth branch advertised as the paper's second contribution. Please report \hat{D} metrics in all depth tables and ablations, and state explicitly whether the SoTA depth claim is intended for \hat{D} or only for \hat{D}_r; if the former, the current evidence does not support it.
  2. [Section 4.4; Supplement B.2.3] The zero-shot DDAD evaluation uses median scaling, which multiplies the predicted depth by the ratio of median ground-truth depth to median predicted depth. This removes the global scale factor and therefore demonstrates scale-agnostic or relative-depth transfer, not metric-depth generalization. The claim of 'exceptional generalizability' in Section 4.4 is consequently overstated. Please report the zero-shot results without median scaling as well, or explicitly re-frame the DDAD experiment as evaluating relative-depth generalization.
  3. [Tables 1, 2; Section 4.3] The occupancy improvements over previous methods are small: Table 1 reports 0.93 versus 0.92 scene accuracy, and Table 2 reports a tie at 0.79 in object accuracy with BTS and KYN. No variance, confidence intervals, or significance tests are reported for any occupancy metric, and the ablation table also appears to be based on a single run. Given the small absolute margins, the SoTA claim for 3D occupancy prediction needs repeated-run statistics or an explicit statement about run-to-run variability; as written, the differences could be within noise.
minor comments (5)
  1. [Section 4.2.1] The text states that 'O_s_acc, Os_rec, and Oo_rec increase by 1.1-3.4%, 4.6-15.0%, and 4.9-30.6%', but Tables 1 and 2 report only O_s_acc, IEs_acc, IEs_rec, Oo_acc, IEo_acc, and IEo_rec; the metric names O_s_rec and Oo_rec are not defined or listed. Please reconcile the metric names and report the actual values.
  2. [Eq. (5)] The term 'Hardmard product' should be 'Hadamard product'.
  3. [Supplement A.2.2] The symbol N is used both for the total number of instances in equation (A.4) and for the number of valid pixels in Eq. (9); please disambiguate these notations.
  4. [Section 3.4 and Supplement A.2.2] The claim that approximately 95.5% of Gaussian samples fall within the bounding box follows from the 2-sigma rule, but this depends on the covariance initialization in Eq. (5); please state explicitly that b_k is the half-width/half-height vector so the correspondence is clear.
  5. [Throughout] Several comparisons in the text cite percentage decreases (e.g., '5.8%' and '5.9%' in Abs Rel) that are consistent with the tables, but there is no explanation of whether the baseline methods are re-run under the same training recipe or taken from their original papers; please clarify the evaluation protocol.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: ViPOcc is a self-supervised empirical pipeline whose predictions are benchmarked against held-out ground truth, and its VFM priors and losses are external inputs rather than fitted versions of the evaluation targets.

full rationale

ViPOcc is an empirical, self-supervised pipeline rather than a closed-form derivation, and I find no step where a claimed prediction reduces by construction to its inputs. The inverse depth alignment module in Eq. (3) learns a residual inverse-depth correction f(F_s, θ) from spatial features; this residual is trained under the temporal photometric loss L_ta (Eq. 9) and the reconstruction consistency loss L_rc (Eq. 10), neither of which regresses to the evaluation ground-truth depth labels. The Depth Anything V2 pseudo-depth prior is an external pretrained model, and the paper explicitly reports its raw performance (0.586 Abs Rel on KITTI-360) as a baseline it improves upon, so the prior is not silently substituted for the claimed result. Occupancy predictions come from a NeRF density field trained with photometric and consistency losses and are evaluated on accumulated LiDAR-based occupancy ground truth. The SNOG sampler is a ray-sampling strategy, not a fitted predictor, and its ablations show independent gains. The zero-shot DDAD result uses median scaling, which is a standard evaluation alignment and is disclosed; it does not constitute circularity. The main caveat is that the supplement (B.1.2) states that depth metrics are evaluated on the rendered depth रेण्डरेड डेपथ है, not on the metric-depth branch output डेपथ है, and no ablation reports डेपथ-only metrics; this is a reporting and attribution limitation for the claimed depth contribution, but it is not circular because the reported rendered-depth numbers are still genuine model predictions against external ground truth. There are no load-bearing self-citations and no imported uniqueness theorems, so the derivation chain is self-contained in the relevant sense.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on pretrained VFM priors and self-supervised losses rather than on new physical entities. The main free parameters are the loss weights and sampling hyperparameters, several of which are not reported by the authors.

free parameters (4)
  • Loss weights lambda_1 and lambda_2 = not reported in main text
    Equation (7) balances temporal alignment and reconstruction consistency; values are chosen empirically and not stated.
  • Background sampling ratio gamma = not reported
    Controls the mixture weight between Gaussian instance sampling and uniform background sampling in Eq. (4); only the formula is given, no chosen value.
  • Patch size l = 8 (from supplement: 64 patches of 8 by 8 pixels)
    Used in the non-overlap constraint Eq. (6) and in sampling; set by experiment.
  • Occupancy threshold tau = 0.5
    Discretizes predicted density into binary occupancy during evaluation; affects all occupancy metrics.
assumptions (5)
  • domain assumption Depth Anything V2 pseudo depth provides a useful metric initialization for KITTI scenes
    Used in Sec. 3.3 and Table 4; raw pseudo depth is poor but its residual is assumed learnable.
  • domain assumption Grounded-SAM with Cityscapes prompts reliably localizes crucial instances such as cars and pedestrians
    Supplies metadata in Sec. 3.4; if detections fail, the sampler falls back to random sampling and the instance-aware gains vanish.
  • domain assumption Photometric consistency across adjacent frames is a valid training signal in driving scenes
    The temporal alignment loss in Eq. (9) assumes Lambertian surfaces and mostly static scenes, as in Monodepth2 and BTS.
  • standard math The NeRF volume rendering model of BTS remains valid for density field prediction
    The framework is built on BTS's density field and volume rendering equations (Eqs. 1 and 2).
  • domain assumption LiDAR-carved occupancy labels from 300-frame sweeps are accurate pseudo ground truth
    Used in evaluation on KITTI-360 (Appendix B.2.1); errors in carving directly affect reported metrics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ViPOcc: Leveraging Visual Priors from Vision Foundation Models for Single-View 3D Occupancy Prediction." pith.science (2026). https://pith.science/paper/UTWLRBQP

@misc{pith2026241211210,
  author       = {Pith},
  title        = {Pith review of: ViPOcc: Leveraging Visual Priors from Vision Foundation Models for Single-View 3D Occupancy Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UTWLRBQP}},
  note         = {Machine review of arXiv:2412.11210}
}
read the original abstract

Inferring the 3D structure of a scene from a single image is an ill-posed and challenging problem in the field of vision-centric autonomous driving. Existing methods usually employ neural radiance fields to produce voxelized 3D occupancy, lacking instance-level semantic reasoning and temporal photometric consistency. In this paper, we propose ViPOcc, which leverages the visual priors from vision foundation models (VFMs) for fine-grained 3D occupancy prediction. Unlike previous works that solely employ volume rendering for RGB and depth image reconstruction, we introduce a metric depth estimation branch, in which an inverse depth alignment module is proposed to bridge the domain gap in depth distribution between VFM predictions and the ground truth. The recovered metric depth is then utilized in temporal photometric alignment and spatial geometric alignment to ensure accurate and consistent 3D occupancy prediction. Additionally, we also propose a semantic-guided non-overlapping Gaussian mixture sampler for efficient, instance-aware ray sampling, which addresses the redundant and imbalanced sampling issue that still exists in previous state-of-the-art methods. Extensive experiments demonstrate the superior performance of ViPOcc in both 3D occupancy prediction and depth estimation tasks on the KITTI-360 and KITTI Raw datasets. Our code is available at: \url{https://mias.group/ViPOcc}.

Figures

Figures reproduced from arXiv: 2412.11210 by the authors.

Figure 1
Figure 1. Single-view 3D scene reconstruction results. KYN [18] struggles to recover clear object boundaries (green boxes) and exhibits poor reconstruction performance for distant objects (blue circles). ViPOcc outperforms KYN in both monocular depth esti￾mation and 3D occupancy prediction tasks. sors [12]. More recently, neural radiance field (NeRF)- based approaches have emerged as promising techniques for unsupervised sing… view at source ↗
Figure 2
Figure 2. An illustration of our proposed ViPOcc framework. Unlike previous approaches that rely solely on NeRF for 3D scene re￾construction, ViPOcc introduces an additional depth prediction branch and an instance-aware SNOG sampler for temporal photometric alignment and spatial geometric alignment. 2.3. Unsupervised Monocular Depth Estimation Existing frameworks typically maximize photometric con￾sistency across video sequen… view at source ↗
Figure 3
Figure 3. An illustration of our proposed SNOG sampler. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison of 3D occupancy prediction on [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Depth distribution comparison. However, it relies on hand-crafted criteria rather than di￾rectly learning the 3D structure from a single view [18]. Qualitative comparisons are presented in [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Comparison of metric depth estimation on the KITTI [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: An illustration of stereo and fisheye images from the KITTI-360 dataset. The fisheye images are resampled using a virtual [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Comparison of sampled patches achieved using our proposed SNOG sampler and the random sampler. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Comparison of 3D occupancy prediction on the KITTI-360 dataset: (a) input RGB images; (b) BTS [ [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Comparison of metric depth estimation on the KITTI-360 dataset: (a) input RGB images; (b) BTS [ [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VISA: VLM-Guided Instance Semantic Auditing for 3D Occupancy World Models

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    VISA improves closed-set 3D occupancy mIoU on nuScenes by using VLM instance audits as reliability-weighted semantic supervisors during training of existing world models.

Reference graph

Works this paper leans on

42 extracted references · 37 canonical work pages · cited by 1 Pith paper

  1. [1]

    Unsupervised Scale-Consistent Depth and Ego-Motion Learning from Monocular Video.Advances in Neural Information Processing Systems (NeurIPS) , 32,

    Jiawang Bian et al. Unsupervised Scale-Consistent Depth and Ego-Motion Learning from Monocular Video.Advances in Neural Information Processing Systems (NeurIPS) , 32,

    work page

  2. [2]

    MonoScene: Monocular 3D Semantic Scene Completion

    Anh-Quan Cao and Raoul De Charette. MonoScene: Monocular 3D Semantic Scene Completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 3991–4001, 2022. 1, 2

    work page 2022

  3. [3]

    Self-Supervised Monocular Depth Esti- mation: Solving the Edge-Fattening Problem

    Xingyu Chen et al. Self-Supervised Monocular Depth Esti- mation: Solving the Edge-Fattening Problem. In Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 5776–5786, 2023. 2

    work page 2023

  4. [4]

    The Cityscapes Dataset for Semantic Ur- ban Scene Understanding

    Marius Cordts et al. The Cityscapes Dataset for Semantic Ur- ban Scene Understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3213–3223, 2016. 4, 11

    work page 2016

  5. [5]

    ImageNet: A large-scale hierarchical im- age database

    Jia Deng et al. ImageNet: A large-scale hierarchical im- age database. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 248–255. IEEE, 2009. 11

    work page 2009

  6. [6]

    Vision Meets Robotics: the KITTI Dataset

    Andreas Geiger et al. Vision Meets Robotics: the KITTI Dataset. The International Journal of Robotics Research, 32 (11):1231–1237, 2013. 5, 13

    work page 2013

  7. [7]

    Digging into Self-Supervised Monoc- ular Depth Estimation

    Cl ´ement Godard et al. Digging into Self-Supervised Monoc- ular Depth Estimation. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision (ICCV) , pages 3828–3838, 2019. 3, 5, 6, 7, 11, 13

    work page 2019

  8. [8]

    3D Packing for Self-Supervised Monocular Depth Estimation

    Guizilini et al. 3D Packing for Self-Supervised Monocular Depth Estimation. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) , pages 2485–2494, 2020. 5, 8, 13

    work page 2020

Show all 42 references
  1. [9]

    Semantically-Guided Representa- tion Learning for Self-Supervised Monocular Depth

    Vitor Guizilini et al. Semantically-Guided Representa- tion Learning for Self-Supervised Monocular Depth. arXiv preprint arXiv:2002.12319, 2020. 2, 3

    2002 arXiv

  2. [10]

    Boosting Self-Supervision for Single- View Scene Completion via Knowledge Distillation

    Keonhee Han et al. Boosting Self-Supervision for Single- View Scene Completion via Knowledge Distillation. In Pro- 8 ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 9837–9847, 2024. 1, 2, 3, 4, 7

    2024

  3. [11]

    Deep Residual Learning for Image Recog- nition

    Kaiming He et al. Deep Residual Learning for Image Recog- nition. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR) , pages 770– 778, 2016. 4, 11

    2016

  4. [12]

    Tri-Perspective View for Vision- Based 3D Semantic Occupancy Prediction

    Yuanhui Huang et al. Tri-Perspective View for Vision- Based 3D Semantic Occupancy Prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 9223–9232, 2023. 1, 2

    2023

  5. [13]

    SelfOcc: Self-Supervised Vision- Based 3D Occupancy Prediction

    Yuanhui Huang et al. SelfOcc: Self-Supervised Vision- Based 3D Occupancy Prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19946–19956, 2024. 1

    2024

  6. [14]

    Fine-Grained Semantics-Aware Representation Enhancement for Self-Supervised Monocu- lar Depth Estimation

    Hyunyoung Jung et al. Fine-Grained Semantics-Aware Representation Enhancement for Self-Supervised Monocu- lar Depth Estimation. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision (ICCV) , pages 12642–12652, 2021. 2

    2021

  7. [15]

    LERF: Language Embedded Radiance Fields

    Justin Kerr et al. LERF: Language Embedded Radiance Fields. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV) , pages 19729–19739,

  8. [16]

    Segment Anything

    Alexander Kirillov et al. Segment Anything. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion (ICCV), pages 4015–4026, 2023. 4, 11

    2023

  9. [17]

    Yang et al

    L. Yang et al. Depth Anything V2. arXiv preprint arXiv:2406.09414, 2024. 2, 4, 6, 12

    2024 arXiv

  10. [18]

    Know Your Neighbors: Improving Single- View Reconstruction via Spatial Vision-Language Reason- ing

    Rui Li et al. Know Your Neighbors: Improving Single- View Reconstruction via Spatial Vision-Language Reason- ing. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 9848– 9858, 2024. 1, 2, 4, 5, 6, 7, 8, 11, 12, 13, 14, 15, 16

    2024

  11. [19]

    KITTI-360: A Novel Dataset and Bench- marks for Urban Scene Understanding in 2D and 3D

    Yiyi Liao et al. KITTI-360: A Novel Dataset and Bench- marks for Urban Scene Understanding in 2D and 3D. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3292–3310, 2022. 5, 12, 15, 16

    2022

  12. [20]

    Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection

    Shilong Liu et al. Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection. In European Conference on Computer Vision , pages 38–55. Springer, 2025. 4, 11

    2025

  13. [21]

    OpenScene: 3D Scene Understanding with Open V ocabularies

    Songyou Peng et al. OpenScene: 3D Scene Understanding with Open V ocabularies. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 815–824, 2023. 2

    2023

  14. [22]

    Grounded SAM: Assembling Open- World Models for Diverse Visual Tasks

    Tianhe Ren et al. Grounded SAM: Assembling Open- World Models for Diverse Visual Tasks. arXiv preprint arXiv:2401.14159, 2024. 2, 4, 6, 12

    2024 arXiv

  15. [23]

    R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras

    Aron Schmied et al. R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 3216–3226, 2023. 3

    2023

  16. [24]

    SwinDepth: Unsuper- vised Depth Estimation Using Monocular Sequences via Swin Transformer and Densely Cascaded Network

    Dongseok Shim and H Jin Kim. SwinDepth: Unsuper- vised Depth Estimation Using Monocular Sequences via Swin Transformer and Densely Cascaded Network. In In- ternational Conference on Robotics and Automation (ICRA), pages 4983–4990. IEEE, 2023. 7

    2023

  17. [25]

    SC-DepthV3: Robust Self-Supervised Monocular Depth Estimation for Dynamic Scenes

    Libo Sun et al. SC-DepthV3: Robust Self-Supervised Monocular Depth Estimation for Dynamic Scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(1):497–508, 2024. 1

    2024

  18. [26]

    Dynamo-Depth: Fixing Unsupervised Depth Estimation for Dynamical Scenes

    Yihong Sun and Bharath Hariharan. Dynamo-Depth: Fixing Unsupervised Depth Estimation for Dynamical Scenes. Ad- vances in Neural Information Processing Systems (NeurIPS), 36, 2024. 3, 7

    2024

  19. [27]

    Occ3D: A large-scale 3d occupancy pre- diction benchmark for autonomous driving

    Xiaoyu Tian et al. Occ3D: A large-scale 3d occupancy pre- diction benchmark for autonomous driving. Advances in Neural Information Processing Systems (NeurIPS), 36, 2024. 1

    2024

  20. [28]

    InternImage: Exploring Large-Scale Vi- sion Foundation Models with Deformable Convolutions

    Wenhai Wang et al. InternImage: Exploring Large-Scale Vi- sion Foundation Models with Deformable Convolutions. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 14408–14419,

  21. [29]

    SQLdepth: Generalizable Self- Supervised Fine-Structured Monocular Depth Estimation

    Youhong Wang et al. SQLdepth: Generalizable Self- Supervised Fine-Structured Monocular Depth Estimation. In Proceedings of the AAAI Conference on Artificial Intelli- gence (AAAI), pages 5713–5721, 2024. 3

    2024

  22. [30]

    The Temporal Opportunist: Self- Supervised Multi-Frame Monocular Depth

    Jamie Watson et al. The Temporal Opportunist: Self- Supervised Multi-Frame Monocular Depth. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 1164–1174, 2021. 3

    2021

  23. [31]

    SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving

    Yi Wei et al. SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 21729–21740, 2023. 1

    2023

  24. [32]

    Behind the Scenes: Density Fields for Single View Reconstruction

    Felix Wimbauer et al. Behind the Scenes: Density Fields for Single View Reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9076–9086, 2023. 1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 14, 15, 16

    2023

  25. [33]

    Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data

    Lihe Yang et al. Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10371–10381, 2024. 2, 6

    2024

  26. [34]

    Learning to Recover 3D Scene Shape from a Single Image

    Wei Yin et al. Learning to Recover 3D Scene Shape from a Single Image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 204–213, 2021. 2

    2021

  27. [35]

    GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose

    Zhichao Yin and Jianping Shi. GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 1983–1992,

    1983

  28. [36]

    PixelNeRF: Neural Radiance Fields from One or Few Images

    Alex Yu et al. PixelNeRF: Neural Radiance Fields from One or Few Images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4578–4587, 2021. 6

    2021

  29. [37]

    OccNeRF: Self-Supervised Multi- Camera Occupancy Prediction with Neural Radiance Fields

    Chubin Zhang et al. OccNeRF: Self-Supervised Multi- Camera Occupancy Prediction with Neural Radiance Fields. arXiv preprint arXiv:2312.09243, 2023. 2

    2023 arXiv

  30. [38]

    Lite-Mono: a Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation

    Ning Zhang et al. Lite-Mono: a Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation. In Proceedings of the IEEE/CVF Confer- 9 ence on Computer Vision and Pattern Recognition (CVPR) , pages 18537–18546, 2023. 7

    2023

  31. [39]

    Vision-Based 3D Occupancy Predic- tion in Autonomous Driving: A Review and Outlook

    Yanan Zhang et al. Vision-Based 3D Occupancy Predic- tion in Autonomous Driving: A Review and Outlook. arXiv preprint arXiv:2405.02595, 2024. 2

    2024 arXiv

  32. [40]

    MonoOcc: Digging into Monocu- lar Semantic Occupancy Prediction

    Yupeng Zheng et al. MonoOcc: Digging into Monocu- lar Semantic Occupancy Prediction. In 2024 IEEE Inter- national Conference on Robotics and Automation (ICRA) , pages 18398–18405, 2024. 2

    2024

  33. [41]

    bridge” and “tunnel

    Tinghui Zhou et al. Unsupervised Learning of Depth and Ego-Motion from Video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1851–1858, 2017. 3 10 A. Additional Methodological Details A.1. Framework Architecture For the paral...

    2017

  34. [42]

    The stereo image pairs with adjacent timestamps are fed into the network for image ren- dering and photometric reconstruction

    using the Eigen split [7]. The stereo image pairs with adjacent timestamps are fed into the network for image ren- dering and photometric reconstruction. Due to the absence of 3D ground-truth labels, we only report the depth estima- tion metrics on the KITTI Raw dataset. B.2.3...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.