Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimation

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read UniGeo claims that a pretrained video diffusion model can be fine-tuned to predict globally consistent surface normals and world coordinates for video, with no camera poses as input.

desk verdict Solid static-scene geometry from video diffusion with a real insight, but the SOTA and dynamic-generalization claims outrun the paper's own tables. read the letter →

arxiv 2505.24521 v1 pith:AGDIUM2M submitted 2025-05-30 cs.CV

Yang-Tian Sun , Xin Yu , Zehuan Huang , Yi-Hua Huang , Yuan-Chen Guo , Ziyi Yang , Yan-Pei Cao , Xiaojuan Qi This is my paper
classification cs.CV
keywords videodiffusiongeometryestimationsurfacenormals3Dreconstructionglobalcoordinatesystemsharedpositionalencodingmulti-tasklearningconsistentprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

UniGeo claims that the inter-frame consistency a video generation model learns for appearance can be repurposed for geometry, as long as the geometric targets obey the same correspondence law: the same 3D point should receive the same predicted value wherever it appears. The paper fine-tunes a pretrained video diffusion transformer to output surface normals and world positions defined in a shared global coordinate system, treating RGB frames and a noisy geometry sequence as one token sequence and reusing RGB positional embeddings for the geometry tokens. On the ScanNet++ benchmark the method matches or beats per-frame baselines that are transformed into the global frame using ground-truth cameras, and on 7Scenes it produces reconstructions competitive with dedicated unposed reconstruction models. If the claim holds, video diffusion priors can serve as a camera-free backbone for consistent geometry estimation that feeds directly into reconstruction.

What carries the argument

The central mechanism is the Shared Positional Encoding (SPE): geometry tokens discard their own positional embeddings and adopt the positional embeddings of the RGB tokens, so tokens at matching spatial and temporal locations attend to each other as correspondences. This is paired with a global-coordinate representation that defines geometry targets in the coordinate frame of the first frame, and with one-step deterministic training that fixes the timestep at $T$, uses zero noise, and trains with an MSE loss. Together these components let the pretrained video diffusion transformer transfer its appearance-based correspondence prior to geometry without architectural modification.

What would settle it

A direct test would take a video with controlled large object motion, train only on static scenes, and compare the predicted global coordinates of a moving object's surface patches across frames against LiDAR or multi-view track ground truth; if the object's predicted position jumps between frames or the normal field breaks under motion, the correspondence-transfer premise fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that the inter-frame correspondence embedded in a pretrained video diffusion model transfers to geometric estimation when the prediction targets share the same correspondence structure as RGB video frames. Predicting normals and positions in a unified global coordinate system, rather than in each frame's camera coordinates, aligns geometric values for the same physical point across frames and lets the model's self-attention propagate consistency. The authors further show that a single network jointly predicting multiple geometric attributes outperforms separately trained networks, and that a one-step deterministic fine-tuning scheme is both cheaper and more accurate than multi-step denoising for this task.

Load-bearing premise

The load-bearing premise is that the correspondence a video generator learned for appearance—same object keeps its color across frames—transfers to predicted geometry when the geometry is expressed in one global frame; the paper supports this with attention visualizations and ablations, but the transfer is not proven from first principles and is least tested for moving objects.

Editorial extensions

If this is right

  • A single fine-tuned video generator can produce globally consistent geometry for multi-view images or video clips without camera poses, so reconstruction pipelines could skip pose estimation and global alignment for static scenes.
  • Joint training on multiple geometric attributes that share the same correspondence improves both tasks, suggesting that mixed geometry-annotation datasets can be pooled in one unified model.
  • The reported generalization to dynamic scenes despite static-only training implies that the appearance-correspondence prior supplies part of the motion signal, potentially extending the method beyond its training distribution.
  • If the central claim is correct, scaling up video generation models should improve geometric consistency as well as visual quality, making video diffusion a competitive base for dense 3D perception.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the correspondence-transfer hypothesis is right, attention alignment between RGB token pairs and geometry token pairs could serve as a diagnostic for where global consistency will fail, providing a testable quantity beyond the reported ablations.
  • The paper's noted error accumulation when stitching short clips suggests a natural extension: add a persistent spatial memory or recurrent state to the global-coordinate formulation so that long videos can be processed without re-alignment.
  • Dynamic-scene generalization could be quantified by evaluating on videos with moving objects and comparing per-object global-coordinate drift against ground-truth motion; such a test would separate genuine motion understanding from a static-scene prior.
  • The global-frame formulation may transfer to other geometric attributes that share inter-frame correspondence, such as semantic occupancy or per-point features, giving a path toward unified video-based scene understanding from one diffusion model.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes UniGeo, a method that fine-tunes a pretrained video diffusion model (CogVideoX-5B) to predict global-coordinate geometric attributes (surface normals and radius, i.e., distance to the global origin) from video or multi-view image inputs without camera parameters. The key ideas are to predict geometry in a shared global coordinate system so that the same 3D point has consistent target values across frames, to condition the model by serially concatenating RGB and geometry tokens and reusing positional embeddings (shared positional encoding, SPE), and to jointly train multiple geometry attributes in one network. Experiments on ScanNet++ and 7Scenes compare UniGeo with per-frame baselines transformed into the global frame using ground-truth cameras and with recent video-depth methods; the method is also applied to reconstruction. A supplementary section reports additional results on static and dynamic datasets, including Bonn.

Significance. If the central claims hold, UniGeo would be a meaningful advance: it shows that video diffusion priors can be repurposed, without architectural changes, for consistent multi-frame geometric estimation in a global frame, and that joint training of multiple geometry attributes with shared correspondence improves accuracy. The paper provides held-out evaluations on real datasets (ScanNet++, 7Scenes), comparisons with external baselines, and ablations of the main components (concatenation strategy, global coordinates, multi-attribute training). These are genuine strengths. However, the paper overstates its results in two load-bearing places: the Table 1 caption claims state-of-the-art performance in both normal and radius estimation, but the table shows UniGeo is not best on all primary metrics; and the dynamic-scene generalization claim is supported only by qualitative results, while the one quantitative dynamic benchmark does not show an advantage. The radius evaluation also relies on per-sequence least-squares alignment, which can absorb global scale drift and weakens the consistency claim.

major comments (4)
  1. [§5.2, Table 1] The claim in the Table 1 caption that UniGeo achieves 'state-of-the-art results in both normal and radius estimation' is not supported by the paper's own numbers. For radius, E2E FT achieves AbsRel of 9.8 versus UniGeo's 10.2; for normal, E2E FT achieves 11.25° accuracy of 65.02 versus UniGeo's 63.38. UniGeo is best on mean/median normal error and δ1 radius, but the blanket state-of-the-art claim is contradicted by the table. Please qualify the claim to name the specific metrics where UniGeo leads (e.g., 'best mean/median normal error and δ1 radius') or add the comparisons that justify the stronger statement.
  2. [§5.4 and Table S1 (Supplementary)] The claimed generalization to dynamic scenes is not established. The main text offers only qualitative comparisons (Fig. 7), while the only quantitative dynamic benchmark, Bonn in Table S1, shows UniGeo's global AbsRel of 8.03 is worse than DepthCrafter's 7.82, and its local AbsRel of 8.09 is worse than DepthCrafter's 7.47. The supplementary text's statement that 'our method still outperforms competitors' on these datasets is inconsistent with these numbers. Since the model is trained exclusively on static data, the abstract and Section 5.4 claim that the video prior supplies the missing motion for dynamic scenes is unsupported. Please either temper the dynamic-scene claim to what the evidence supports or provide a quantitative comparison that demonstrates an advantage on a dynamic benchmark.
  3. [§5.2, radius evaluation protocol] The radius evaluation aligns the predicted radius with ground truth by per-sequence least-squares fitting before computing AbsRel, RMSE, and δ1. This per-sequence alignment can absorb global scale and offset errors, which is precisely the kind of drift that a global-consistency method should be held accountable for. Since UniGeo's central advantage is consistency across frames in a single global coordinate system, the evaluation should also report a metric that does not per-sequence align the radius (e.g., global scale-invariant error without per-sequence fitting), or clearly justify that the alignment is standard practice and that the consistency claim is separately demonstrated by the reconstruction experiments.
  4. [§5.3, Table 2] The comparison with Dust3R removes Dust3R's final alignment step 'for a fair comparison' because UniGeo does not use such a step. However, global alignment is an integral part of Dust3R's method for producing a unified reconstruction; removing it handicaps the baseline and makes the 'comparable performance to Spann3R and significantly outperforms Dust3R' claim less convincing. Please either report Dust3R with its standard alignment, or apply the same alignment post-processing to all methods and state this explicitly in the protocol.
minor comments (4)
  1. [Throughout] There are several typos and grammatical errors: 'Parameters θ' in Section 3, 'estiamtion' in Section 5.2, 'seperate' in Section 5.5, and 'the final half of the tokens' which should be 'the second half'. Please proofread the manuscript.
  2. [References] Some references are duplicated: [15] and [16] both refer to GeoWizard, [20] and [21] are the same DDPM paper, and [6] and [7] are the same Stable Video Diffusion paper. Please consolidate duplicate entries.
  3. [§4.4] The one-step deterministic training description could be clarified: it says the timestep is fixed at t=T and noise is set to zero, but does not specify how the model's timestep embedding is handled or how inference is performed (presumably a single forward pass). Adding this detail would help reproducibility.
  4. [§5.5, Table 3] The 'w/o Global Coord' ablation predicts geometry in each frame's camera coordinate system, but the evaluation protocol for this variant is not described. It should be stated whether these predictions are transformed to the global frame with ground-truth cameras before computing the reported metrics; otherwise the comparison is ambiguous.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: UniGeo's core claims are validated on external benchmarks with ablations, and no load-bearing step reduces by construction to its inputs or to self-citation.

full rationale

The paper's central derivation—that fine-tuning a pretrained video diffusion model on global-coordinate geometry targets yields consistent cross-frame geometry—is not circular. The global-coordinate representation is a supervised training target defined from ground-truth camera poses (Eqs. 5–7 in the supplement), not a quantity fitted from the evaluation data or renamed from the method's own outputs. The claimed benefit of the video prior over per-frame baselines is tested against external methods on held-out real datasets (ScanNet++, 7Scenes, DAVIS, Bonn), and the key design choices (global coordinate system, shared positional encoding, multi-attribute joint training) are isolated by the ablation study in Table 3 with the no-global-coordinate and no-SPE variants performing worse. The radius evaluation uses per-sequence least-squares alignment to ground truth, but this is a standard monocular-depth evaluation protocol explicitly attributed to prior work [10, 26] and does not enter the trained model or the derivation of the consistency claim. The dynamic-scene generalization claim is supported only qualitatively in the main text and is quantitatively weaker on the Bonn benchmark in Table S1, but this is a correctness/scoping concern, not a circularity: the model is not defined in terms of the dynamic-scene results. The stated limitations in Sec. 6 (limited sequence length, accumulated stitching errors, high computational cost, occasional blur) are practical shortcomings and do not reveal any circular dependency. No load-bearing self-citation chain, imported uniqueness theorem, or ansatz-by-citation was found; the few self-citations (e.g., Geonet/Geonet++ [39, 40]) are historical related-work references and carry no weight in the derivation. The paper's core result is therefore self-contained against external benchmarks and does not reduce to its own inputs by construction.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The evaluation protocol injects one fitted scale per sequence (radius), and the method leans on unproven domain assumptions about correspondence transfer, VAE encoding of geometry, one-step distillation, and dynamic-scene validity of the static global frame. No invented physical entities are introduced.

free parameters (2)
  • Per-sequence radius scale alignment = one least-square scale per evaluation sequence
    Radius predictions are aligned to ground truth by least-square fitting before AbsRel and δ1 are computed (Sec 5.2, following [10, 26]). This fitted scale is part of how the global radius claim is substantiated; the model's metric scale is not independently verified.
  • Multi-view grouping window (NumView) = unspecified in main text; supplementary selects top NumView-1 overlapping views
    The training grouping of Hypersim frames depends on a chosen number of views per group; the value is not reported, which affects training-data composition but not the core inference mechanism.
assumptions (4)
  • domain assumption Pre-trained video diffusion self-attention encodes inter-frame pixel correspondence, and this correspondence transfers to a geometry sequence when the geometry is defined in a global coordinate frame (Sec 4.1, Fig 2).
    This is the paper's central hypothesis. It is motivated by an attention visualization (Fig 2a) and supported by ablations and final performance, but it is not proven or quantitatively isolated.
  • domain assumption The pretrained video VAE encodes geometry maps (normals, world coordinates) into the shared latent space without losing the information needed for accurate prediction (Sec 4.3, Fig 3).
    Geometry maps are encoded with an RGB-trained VAE; the paper does not report VAE reconstruction error for geometry, so the prediction ceiling set by the VAE is unverified.
  • domain assumption A single-step deterministic mapping (fixed t=T, zero noise) is sufficient to learn the geometry estimation function (Sec 4.4).
    Borrowed from image-domain E2E FT [17], extended to video; the paper's stated quantitative support (Sec 5.5) is not present in Table 3, and only qualitative supplement results (Fig S4) are shown.
  • domain assumption For dynamic scenes, the first-frame global coordinate frame remains a valid reference for moving objects, and the static-trained model can infer object motion (abstract, Sec 5.4).
    The model is trained only on static data; global coordinates of moving objects change over time, so dynamic generalization requires the video prior to supply motion. The main text shows only qualitative DAVIS results; quantitative dynamic results appear only in the supplement (Table S1, Bonn).

how reviews work

0 comments
Cite this review

Pith. "Pith review of UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimation." pith.science (2026). https://pith.science/paper/AGDIUM2M

@misc{pith2026250524521,
  author       = {Pith},
  title        = {Pith review of: UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AGDIUM2M}},
  note         = {Machine review of arXiv:2505.24521}
}
read the original abstract

Recently, methods leveraging diffusion model priors to assist monocular geometric estimation (e.g., depth and normal) have gained significant attention due to their strong generalization ability. However, most existing works focus on estimating geometric properties within the camera coordinate system of individual video frames, neglecting the inherent ability of diffusion models to determine inter-frame correspondence. In this work, we demonstrate that, through appropriate design and fine-tuning, the intrinsic consistency of video generation models can be effectively harnessed for consistent geometric estimation. Specifically, we 1) select geometric attributes in the global coordinate system that share the same correspondence with video frames as the prediction targets, 2) introduce a novel and efficient conditioning method by reusing positional encodings, and 3) enhance performance through joint training on multiple geometric attributes that share the same correspondence. Our results achieve superior performance in predicting global geometric attributes in videos and can be directly applied to reconstruction tasks. Even when trained solely on static video data, our approach exhibits the potential to generalize to dynamic video scenes.

Figures

Figures reproduced from arXiv: 2505.24521 by the authors.

Figure 1
Figure 1. UniGeo utilizes video diffusion models to jointly estimate geometric properties—such as surface normals and coordinates [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our key insights lie in (a) pre-trained video diffusion [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Method overview. Our method targets at predicting geometric properties that are defined in the global coordinate system, where “radius” represents the distance from a 3D points to the origin (left). We efficiently adapt a pre-trained video diffusion model that inherently encodes inter-frame correspondence into a consistent geometry estimation model (right), where we process both rgb sequence and geometry sequence th… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative comparisons on normal estimation. Our method produces more consistent normals, remains robust to highlight reflections, and generates results closest to the GT. It effectively removes noise from the GT, delivering smooth predictions. scanner) and IMU camera…
Figure 5
Figure 5. Figure 5: Qualitative comparisons on radius estimation. Our method achieves both accuracy and consistency, producing results closest to the ground truth. The entire video is normalized for consistent visualization. Methods Normal Radius Mean↓ Median↓ 11.25↑ AbsRel↓ δ1↑ Marigold …
Figure 6
Figure 6. Figure 6: Comparison to Dust3R. Our method demonstrates bet￾ter inter-frame consistency in multi-frame sequence reconstruc￾tion. Please refer to the videos in Supp for clearer comparisons. Input E2E FT Ours [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Comparison to E2E. Our method predicts normals of better consistency with more details preserved . estimation in dynamic scenes. We present some estimation results on DAVIS videos [38], along with comparisons to single-frame estimation methods. For additional compar￾is…

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models

    cs.CV 2025-06 conditional novelty 6.0 of 10

    E3D-Bench compares 16 3D geometric foundation models on depth, reconstruction, pose, and view-synthesis tasks with a unified evaluation toolkit.

  2. Reconstructing 4D Spatial Intelligence: A Survey

    cs.CV 2025-07 accept novelty 4.0 of 10

    A review that classifies 4D scene reconstruction methods into five progressive levels: low-level cues, scene components, dynamic scenes, interactions, and physics.

Reference graph

Works this paper leans on

67 extracted references · 28 canonical work pages · cited by 2 Pith papers

  1. [1]

    https://hailuoai.video, 2023

    Hailuo. https://hailuoai.video, 2023. 3

    work page 2023

  2. [2]

    https://kling.kuaishou.com, 2023

    Keling. https://kling.kuaishou.com, 2023. 3

    work page 2023

  3. [3]

    www.imagine.art, 2023

    Pika. www.imagine.art, 2023. 3

    work page 2023

  4. [4]

    Rethinking induc- tive biases for surface normal estimation

    Gwangbin Bae and Andrew J Davison. Rethinking induc- tive biases for surface normal estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9535–9545, 2024. 3

    work page 2024

  5. [5]

    Es- timating and exploiting the aleatoric uncertainty in surface normal estimation

    Gwangbin Bae, Ignas Budvytis, and Roberto Cipolla. Es- timating and exploiting the aleatoric uncertainty in surface normal estimation. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 13137–13146,

    work page

  6. [7]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 3

    arXiv 2023

  7. [8]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, et al. Video generation models as world simulators. OpenAI Blog, 1:8, 2024. 3

    work page 2024

  8. [10]

    Video depth anything: Consistent depth estimation for super-long videos

    Sili Chen, Hengkai Guo, Shengnan Zhu, Feihu Zhang, Zi- long Huang, Jiashi Feng, and Bingyi Kang. Video depth anything: Consistent depth estimation for super-long videos. arXiv:2501.12375, 2025. 2, 6, 7

    arXiv 2025

Show all 67 references
  1. [11]

    Single- image depth perception in the wild

    Weifeng Chen, Zhao Fu, Dawei Yang, and Jia Deng. Single- image depth perception in the wild. Advances in neural in- formation processing systems, 29, 2016. 3

    2016

  2. [12]

    Omnidata: A scalable pipeline for making multi- task mid-level vision datasets from 3d scans

    Ainaz Eftekhar, Alexander Sax, Jitendra Malik, and Amir Zamir. Omnidata: A scalable pipeline for making multi- task mid-level vision datasets from 3d scans. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 10786–10796, 2021. 3

    2021

  3. [13]

    Predicting depth, surface nor- mals and semantic labels with a common multi-scale con- volutional architecture

    David Eigen and Rob Fergus. Predicting depth, surface nor- mals and semantic labels with a common multi-scale con- volutional architecture. In Proceedings of the IEEE inter- national conference on computer vision , pages 2650–2658,

  4. [14]

    Depth map prediction from a single image using a multi-scale deep net- work

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep net- work. Advances in neural information processing systems , 27, 2014. 3

    2014

  5. [15]

    Geowiz- ard: Unleashing the diffusion priors for 3d geometry estima- tion from a single image

    Xiao Fu, Wei Yin, Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojie Shen, Dahua Lin, and Xiaoxiao Long. Geowiz- ard: Unleashing the diffusion priors for 3d geometry estima- tion from a single image. ArXiv, abs/2403.12013, 2024. 6, 7

    2024 arXiv

  6. [16]

    Geowiz- ard: Unleashing the diffusion priors for 3d geometry esti- mation from a single image

    Xiao Fu, Wei Yin, Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojie Shen, Dahua Lin, and Xiaoxiao Long. Geowiz- ard: Unleashing the diffusion priors for 3d geometry esti- mation from a single image. In European Conference on Computer Vision, pages 241–258. Springer, 2024. 1, 2...

    2024

  7. [17]

    Fine-tuning image-conditional diffusion models is easier than you think

    Gonzalo Martin Garcia, Karim Abou Zeid, Christian Schmidt, Daan de Geus, Alexander Hermans, and Bastian Leibe. Fine-tuning image-conditional diffusion models is easier than you think. ArXiv, abs/2409.11355, 2024. 5, 7, 1

    2024

  8. [18]

    Vision meets robotics: The kitti dataset

    Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. The in- ternational journal of robotics research, 32(11):1231–1237,

  9. [19]

    Depthfm: Fast monocular depth estimation with flow matching

    Ming Gui, Johannes Schusterbauer, Ulrich Prestel, Pingchuan Ma, Dmytro Kotovenko, Olga Grebenkova, Stefan Andreas Baumann, Vincent Tao Hu, and Bj ¨orn Ommer. Depthfm: Fast monocular depth estimation with flow matching. arXiv preprint arXiv:2403.13788, 2024. 3

    2024 arXiv

  10. [20]

    Jonathan Ho, Ajay Jain, and P. Abbeel. Denoising diffusion probabilistic models. ArXiv, abs/2006.11239, 2020. 3

    2006 arXiv

  11. [21]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3

    2020

  12. [22]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205.15868, 2022. 3, 6

    2022 arXiv

  13. [24]

    Depthcrafter: Generating consistent long depth sequences for open-world videos

    Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, and Ying Shan. Depthcrafter: Generating consistent long depth sequences for open-world videos. arXiv preprint arXiv:2409.02095, 2024. 2, 3

    2024 arXiv

  14. [25]

    Stereo4d: Learning how things move in 3d from internet stereo videos.arXiv preprint arXiv:2412.09621, 2024

    Linyi Jin, Richard Tucker, Zhengqi Li, David Fouhey, Noah Snavely, and Aleksander Holynski. Stereo4d: Learning how things move in 3d from internet stereo videos.arXiv preprint arXiv:2412.09621, 2024. 3

    2024 arXiv

  15. [26]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9492–9502, 2023...

    2024

  16. [27]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9492...

    2024

  17. [28]

    Hunyuanvideo: A systematic framework for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024. 3 9

    2024 arXiv

  18. [29]

    Ex- ploiting diffusion prior for generalizable dense prediction

    Hsin-Ying Lee, Hung-Yu Tseng, and Ming-Hsuan Yang. Ex- ploiting diffusion prior for generalizable dense prediction. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 7861–7871, 2024. 3

    2024

  19. [30]

    Depth and surface normal estimation from monocular images using regression on deep features and hi- erarchical crfs

    Bo Li, Chunhua Shen, Yuchao Dai, Anton Van Den Hengel, and Mingyi He. Depth and surface normal estimation from monocular images using regression on deep features and hi- erarchical crfs. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1119–1127,

  20. [31]

    Interiornet: Mega-scale multi- sensor photo-realistic indoor scenes dataset

    Wenbin Li, Sajad Saeedi, John McCormac, Ronald Clark, Dimos Tzoumanikas, Qing Ye, Yuzhong Huang, Rui Tang, and Stefan Leutenegger. Interiornet: Mega-scale multi- sensor photo-realistic indoor scenes dataset. In British Ma- chine Vision Conference (BMVC), 2018. 5

    2018

  21. [32]

    Matrixcity: A large-scale city dataset for city-scale neural rendering and beyond

    Yixuan Li, Lihan Jiang, Linning Xu, Yuanbo Xiangli, Zhen- zhi Wang, Dahua Lin, and Bo Dai. Matrixcity: A large-scale city dataset for city-scale neural rendering and beyond. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3205–3215, 2023. 5

    2023

  22. [33]

    Prompting depth anything for 4k resolution accurate metric depth estimation

    Haotong Lin, Sida Peng, Jingxiao Chen, Songyou Peng, Ji- aming Sun, Minghuan Liu, Hujun Bao, Jiashi Feng, Xiaowei Zhou, and Bingyi Kang. Prompting depth anything for 4k resolution accurate metric depth estimation. arXiv preprint arXiv:2412.14015, 2024. 3

    2024

  23. [34]

    Align3r: Aligned monocular depth estimation for dynamic videos

    Jiahao Lu, Tianyu Huang, Peng Li, Zhiyang Dou, Cheng Lin, Zhiming Cui, Zhen Dong, Sai-Kit Yeung, Wenping Wang, and Yuan Liu. Align3r: Aligned monocular depth estimation for dynamic videos. arXiv preprint arXiv:2412.03079, 2024. 3

    2024 arXiv

  24. [35]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 4195–4205,

  25. [36]

    Peebles and Saining Xie

    William S. Peebles and Saining Xie. Scalable diffusion mod- els with transformers. 2023 IEEE/CVF International Confer- ence on Computer Vision (ICCV), pages 4172–4182, 2022. 3

    2023

  26. [37]

    Unidepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10106–10116, 2024. 1, 3

    2024

  27. [38]

    The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017

    Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Ar- bel´aez, Alex Sorkine-Hornung, and Luc Van Gool. The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017. 8

    2017 arXiv

  28. [39]

    Geonet: Geometric neural network for joint depth and surface normal estimation

    Xiaojuan Qi, Renjie Liao, Zhengzhe Liu, Raquel Urtasun, and Jiaya Jia. Geonet: Geometric neural network for joint depth and surface normal estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 283–291, 2018. 3

    2018

  29. [40]

    Geonet++: Iterative geo- metric neural network with edge-aware refinement for joint depth and surface normal estimation

    Xiaojuan Qi, Zhengzhe Liu, Renjie Liao, Philip HS Torr, Raquel Urtasun, and Jiaya Jia. Geonet++: Iterative geo- metric neural network with edge-aware refinement for joint depth and surface normal estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(2...

  30. [41]

    Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer

    Ren ´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE transactions on pattern analysis and machine intelligence, 44(3):1623–1637, 2020. 3

    2020

  31. [42]

    Vi- sion transformers for dense prediction

    Ren ´e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction. In Proceedings of the IEEE/CVF international conference on computer vision, pages 12179–12188, 2021. 3

    2021

  32. [43]

    Hypersim: A photorealis- tic synthetic dataset for holistic indoor scene understanding

    Mike Roberts and Nathan Paczan. Hypersim: A photorealis- tic synthetic dataset for holistic indoor scene understanding. 2021 IEEE/CVF International Conference on Computer Vi- sion (ICCV), pages 10892–10902, 2020. 5

    2021

  33. [44]

    Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer

    Robin Rombach, A. Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10674–10685, 2021. 3

    2022

  34. [45]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 3

    2022

  35. [46]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. ArXiv, abs/1505.04597, 2015. 3

    2015 arXiv

  36. [47]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, pa...

    2015

  37. [48]

    Learning tem- porally consistent video depth from video diffusion priors

    Jiahao Shao, Yuanbo Yang, Hongyu Zhou, Youmin Zhang, Yujun Shen, Matteo Poggi, and Yiyi Liao. Learning tem- porally consistent video depth from video diffusion priors. ArXiv, abs/2406.01493, 2024. 2, 3, 4

    2024 arXiv

  38. [49]

    Scene intrinsics and depth from a single image

    Evan Shelhamer, Jonathan T Barron, and Trevor Darrell. Scene intrinsics and depth from a single image. In Proceed- ings of the IEEE International Conference on Computer Vi- sion Workshops, pages 37–44, 2015. 3

    2015

  39. [50]

    Scene coordinate regression forests for camera relocalization in rgb-d images

    Jamie Shotton, Ben Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, and Andrew William Fitzgibbon. Scene coordinate regression forests for camera relocalization in rgb-d images. 2013 IEEE Conference on Computer Vision and Pattern Recognition, pages 2930–2937, 2013. 5

    2013

  40. [51]

    Indoor segmentation and support inference from rgbd images

    Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In Computer Vision–ECCV 2012: 12th Eu- ropean Conference on Computer Vision, Florence, Italy, Oc- tober 7-13, 2012, Proceedings, Part V 12 , pages 746–760...

    2012

  41. [52]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xiaoyue Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-a-video: Text-to-video generation without text-video data. ArXiv, abs/2209.14792, 2022. 5

    2022 arXiv

  42. [53]

    Deep unsupervised learning using 10 nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using 10 nonequilibrium thermodynamics. In International confer- ence on machine learning, pages 2256–2265. pmlr, 2015. 3

    2015

  43. [54]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. arXiv preprint arXiv:2011.13456, 2020. 3

    2011 arXiv

  44. [55]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Yu Lu, Shengfeng Pan, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. ArXiv, abs/2104.09864, 2021. 6

    2021 arXiv

  45. [56]

    Mv-dust3r+: Single-stage scene reconstruction from sparse views in 2 seconds

    Zhenggang Tang, Yuchen Fan, Dilin Wang, Hongyu Xu, Rakesh Ranjan, Alexander Schwing, and Zhicheng Yan. Mv-dust3r+: Single-stage scene reconstruction from sparse views in 2 seconds. arXiv preprint arXiv:2412.06974, 2024. 3

    2024 arXiv

  46. [58]

    3d reconstruction with spatial memory

    Hengyi Wang and Lourdes Agapito. 3d reconstruction with spatial memory. arXiv preprint arXiv:2408.16061, 2024. 3

    2024 arXiv

  47. [59]

    Continuous 3d perception model with persistent state

    Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state. arXiv preprint arXiv:2501.12387, 2025. 3

    2025 arXiv

  48. [60]

    Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision

    Ruicheng Wang, Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. arXiv preprint arXiv:2410.19115, 2024. 3

    arXiv

  49. [61]

    Dust3r: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and J ´erˆome Revaud. Dust3r: Geometric 3d vision made easy. 2024 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 20697– 20709, 2023. 7

    2024

  50. [62]

    Dust3r: Geometric 3d vi- sion made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vi- sion made easy. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20697– 20709, 2024. 3

    2024

  51. [63]

    Monocular relative depth percep- tion with web stereo data supervision

    Ke Xian, Chunhua Shen, Zhiguo Cao, Hao Lu, Yang Xiao, Ruibo Li, and Zhenbo Luo. Monocular relative depth percep- tion with web stereo data supervision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 311–320, 2018. 3

    2018

  52. [65]

    What matters when repurposing diffusion models for general dense perception tasks? arXiv preprint arXiv:2403.06090 ,

    Guangkai Xu, Yongtao Ge, Mingyu Liu, Chengxiang Fan, Kangyang Xie, Zhiyue Zhao, Hao Chen, and Chunhua Shen. What matters when repurposing diffusion models for general dense perception tasks? arXiv preprint arXiv:2403.06090 ,

    arXiv

  53. [66]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10371–10381, 2024. 1, 3

    2024

  54. [67]

    Depth any- thing v2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth any- thing v2. Advances in Neural Information Processing Sys- tems, 37:21875–21911, 2025. 1, 3

    2025

  55. [68]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, Da Yin, Xiaotao Gu, Yuxuan Zhang, Weihan Wang, Yean Cheng, Ting Liu, Bin Xu, Yuxiao Dong, and Jie Tang. Cogvideox: Text-to-video diffusion models ...

    2024 arXiv

  56. [69]

    Stablenormal: Reducing diffusion variance for stable and sharp normal.ACM Transactions on Graphics,

    Chongjie Ye, Lingteng Qiu, Xiaodong Gu, Qi Zuo, Yushuang Wu, Zilong Dong, Liefeng Bo, Yuliang Xiu, and Xiaoguang Han. Stablenormal: Reducing diffusion variance for stable and sharp normal.ACM Transactions on Graphics,

  57. [70]

    Scannet++: A high-fidelity dataset of 3d in- door scenes

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high-fidelity dataset of 3d in- door scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12–22, 2023. 5

    2023

  58. [71]

    Metric3d: Towards zero-shot metric 3d prediction from a single image

    Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from a single image. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9043–9053, 2023. 1

    2023

  59. [72]

    ScanNet”, “Neural RGBD

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jam- pani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming- Hsuan Yang. Monst3r: A simple approach for estimat- ing geometry in the presence of motion. arXiv preprint arXiv:2410.03825, 2024. 3 11 UniGeo: Taming Video Diffus...

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.