Pith. sign in

REVIEW 3 major objections 6 minor 105 references

Self-Supervised Monocular 4D Scene Reconstruction for Egocentric Videos

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read EgoMono4D claims that a single feed-forward model trained without labels can reconstruct dense, dynamic 4D scenes from egocentric video, and reports that it outperforms supervised and modular baselines.

desk verdict A solid self-supervised baseline for egocentric 4D reconstruction whose headline gain is real but driven mainly by static-scene consistency; the paper is honest about this and deserves a serious referee. read the letter →

arxiv 2411.09145 v4 pith:X5AAQ6QL submitted 2024-11-14 cs.CV cs.RO

Chengbo Yuan , Geng Chen , Li Yi , Yang Gao This is my paper · ORCID
classification cs.CVcs.RO
keywords self-supervisedlearningegocentricvideo4Dscenereconstructionmonoculardepthestimationcameraposepointcloudsequencezero-shotgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EgoMono4D claims that dense 4D reconstruction of egocentric video can be learned without any ground-truth depth, camera pose, or intrinsic labels. Starting from a pretrained single-frame depth-and-intrinsic model, it adds temporal adaptors and a confidence-mask decoder, then trains on 11.2 million unlabeled egocentric frames by aligning multi-frame point clouds under photometric and regularization losses. The paper reports that this single feed-forward model reconstructs dense per-frame point-cloud sequences better than supervised and modular baselines in both in-domain tests (H2O, HOI4D) and zero-shot tests (POV-Surgery, ARCTIC-HOI). The authors position the work as the first self-supervised point-cloud-sequence reconstruction method for label-scarce egocentric video, with the gain coming from joint 3D consistency of depth, intrinsics, and poses rather than from per-variable accuracy.

What carries the argument

The load-bearing mechanism is the conversion of camera-pose estimation into a dense prediction problem. EgoMono4D reframes pose recovery as weighted Procrustes alignment: per-frame depths and intrinsics are unprojected to point clouds, optical flow or long-term tracks give 3D point correspondences, and the best camera transform is the closed-form SVD solution of $\hat{P}_{i,i-1} = \arg\min_{P \in SE(3)} ||\hat{M}_{i,i-1}(\hat{X}_{i-1} - P \hat{X}_{i}^{i-1})||$, with a predicted confidence mask $\hat{M}$ excluding dynamic and unreliable pixels. This is trained end-to-end with the loss $L = \alpha L_{\text{shape}} + \beta L_{\text{flow}} + \gamma L_{\text{track}} + \lambda L_{\text{mask}} + \mu L_{\text{con}}$, where $L_{\text{shape}}$ regularizes each frame's point-cloud shape against pseudo point clouds under a scaled SE(3) alignment, $L_{\text{flow}}$ and $L_{\text{track}}$ enforce 3D consistency in high-confidence regions, and $L_{\text{con}}$ keeps intrinsics consistent across clips. The video backbone is a pretrained single-frame depth model extended with adaptor blocks—a temporal Transformer over global tokens and a U-Net over patch features—plus a new confidence-mask decoder. The pseudo-mask prior comes from hand-object segmentation and flying-pixel edges.

What would settle it

Render or capture a synthetic egocentric video with ground-truth 3D labels in which the manipulated object's true size is deliberately scaled away from what the pretrained depth model predicts, then measure whether EgoMono4D's reconstructed dynamic part inherits that scale error while the static scene stays aligned; if the error disappears when the shape-regularization loss $L_{\text{shape}}$ is ablated, that loss is the carrier of the error.

Watch

Extended reading notes

Core claim

The central claim is that one feed-forward model, EgoMono4D, can jointly estimate camera intrinsics, camera poses, and video depth from monocular egocentric video and turn them into a dense point-cloud sequence in a global coordinate system. Camera poses are not predicted by a separate head; they are solved in closed form by weighted Procrustes alignment of unprojected depths using optical flow, with a learned confidence mask suppressing dynamic regions, occlusions, and unreliable flow. Training is fully self-supervised on unlabeled video: a photometric loss aligns 3D reprojections induced by optical-flow correspondences and long-term tracks, a shape-regularization loss ties the predictions to pseudo point clouds from a pretrained depth model, and mask and intrinsic-consistency losses stabilize training. On two in-domain and two zero-shot egocentric datasets, the paper reports lower Chamfer distance and higher F-scores than DUSt3R, MonSt3R, Align3R, CUT3R, and a SLAM-plus-depth pipeline, and also reports improved long-term 3D scene-flow recovery. The paper explicitly notes that its individual depth and pose estimates are only comparable to baselines; the improvement comes from making the variables mutually consistent in 3D space.

Load-bearing premise

The load-bearing premise is that the pretrained depth model's per-frame point clouds are trustworthy enough, especially the relative size of hands and manipulated objects versus the static scene, to serve as pseudo-labels for training; the paper's Appendix J concedes that when that relative size is wrong, EgoMono4D inherits the distortion.

Editorial extensions

If this is right

  • A dense 4D reconstructor for egocentric video can be trained from unlabeled video alone, removing the labeled-data bottleneck that limits supervised methods in this domain.
  • Jointly optimizing depth, intrinsics, and poses for 3D consistency yields better point-cloud-sequence reconstruction than optimizing each variable independently, even when per-variable depth and pose accuracy is not state of the art.
  • Zero-shot reconstruction transfers to unseen egocentric domains, including surgical video with synthetic textures and hand-object close-ups, at about 0.218 seconds per frame.
  • The same predictions support long-term 3D scene flow, giving trajectory-level output usable for robot learning and hand-object interaction analysis.
  • Frame-rate robustness holds down to about a quarter of the original frame rate, with degradation at very low frame rates traced to the optical-flow module rather than the depth or pose predictors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the central claim holds, the practical ceiling is set by the pretrained depth model's relative-scale accuracy; a stronger pseudo-label depth prior or any sparse metric supervision should directly shrink the dynamic-part size distortion and static misalignment the paper acknowledges in its limitations.
  • Since none of the losses is egocentric-specific, the same training recipe should transfer to general dynamic scenes whenever motion or foreground masks are available; the main missing ingredient is a mask prior analogous to the hand-object segmentation used here.
  • The paper's own depth and pose numbers suggest that 4D reconstruction benchmarks should reward joint consistency rather than component accuracy, and judging methods by end-to-end point-cloud metrics may change which approaches look competitive.
  • A testable extension is to replace the frozen external optical-flow module with in-network correspondences, which should extend the operating range to lower frame rates and remove the current low-frame-rate failure mode.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces EgoMono4D, a feed-forward model that jointly estimates camera intrinsics, camera poses, and video depth from unlabeled egocentric videos, producing dense per-frame pointcloud sequences. The architecture extends the supervised single-frame model UniDepth with video adaptors and a confidence-mask decoder, and is trained with self-supervised photometric losses, a shape-regularization loss using UniDepth pseudo-labels, a mask-regularization loss using EgoHOS, and a camera-consistency term. The authors report that EgoMono4D outperforms all baselines on dense pointclouds sequence reconstruction in both in-domain (H2O, HOI4D) and zero-shot (POV-Surgery, ARCTIC-HOI) settings, and also leads on long-term 3D scene flow recovery. Ablations and additional analyses are provided, and code, models, and visualizations are released.

Significance. If the claimed result holds with proper statistical validation, EgoMono4D is a meaningful step toward label-free 4D reconstruction of egocentric video: a single fast feed-forward model can produce dense, temporally consistent pointcloud sequences without ground-truth depth or pose labels. The paper is commendable for releasing code and models, for including a thorough ablation study (Table 2), and for openly documenting failure modes in Appendix J. The contribution is, however, partly inherited from supervised teachers (UniDepth and EgoHOS), and the evidence suggests the main gain over baselines is in static-scene temporal consistency rather than in dynamic hand-object geometry, where performance is only comparable to DS+UniDepth (Table 6). The approach is nevertheless a credible and reproducible baseline for future self-supervised egocentric 4D reconstruction.

major comments (3)
  1. [§6.1, Table 1; §5.1] Table 1 reports the central 'superior performance' claim as single point estimates with no error bars or significance tests, and §5.1 states that zero-shot evaluation uses only 'the first record from the first participant in each task' for POV-Surgery and ARCTIC-HOI. This is a sample of one per zero-shot dataset, and several margins over the strongest baseline are small (e.g., ARCTIC-HOI CD 2.8 vs 2.9; F1 24.1 vs 22.2). The claim of superiority across all datasets is therefore not statistically substantiated; the authors should report variance across records/participants or otherwise justify that the chosen sub-clips are representative.
  2. [§4.3.2, Eq. (5); Appendix J; Table 6] The shape-regularization loss Lshape aligns the predicted per-frame pointclouds to UniDepth pseudo-labels, so the relative size and shape of dynamic parts is inherited from UniDepth's single-frame predictions. Appendix J explicitly concedes the resulting 'dynamic part (size) distortion' and 'static part misalignment.' Table 6 then shows that on hand-object-only long-term 3D flow, EgoMono4D is only comparable to DS+UniDepth (e.g., POV-Surgery P5 9.9 vs 9.9; HOI4D ADE 76.6 vs 79.3). The headline gain in Table 1 is therefore dominated by static-scene temporal consistency, and the abstract's and introduction's framing of the contribution as superior 4D dynamics is overbroad. Please add a hand-object-focused evaluation metric or qualify the claim to make explicit that the main improvement is in static regions.
  3. [Appendix G, Table 8; §6.1] Appendix G shows that on POV-Surgery, EgoMono4D does not improve per-frame depth (AbsRel 12.6 vs UniDepth's 11.9) or camera pose (ATE 11.54 vs Align3R's 6.35). The improvement in the dense pointclouds sequence metric (Table 1) therefore comes from cross-frame consistency rather than more accurate single-view geometry. This is a valid contribution, but it is only discussed in an appendix; the main text and abstract should state this explicitly and discuss the risk that the global best-aligned SE(3) evaluation (Appendix C.1) rewards consistency at the expense of per-frame accuracy, particularly in dynamic regions.
minor comments (6)
  1. [Abstract, §1, §4] The phrase 'an self-supervised' appears multiple times; it should be 'a self-supervised.'
  2. [§3.2, §4.1, §4.2] Typos: 'arbitray' should be 'arbitrary,' 'derivated' should be 'derived,' and 'Secntion' should be 'Section.'
  3. [§4.3.1, Eq. (4)] The denominator contains F(\hat{X}_j), but the text states 'We use F(\hat{X}_i) as a proxy for the scale of \hat{X}_t'; please clarify which pointcloud is used for the scale normalization.
  4. [§6.4, Table 3] The row labels 'fps / 1', 'fps / 2', etc. are unclear; please define the sampling ratio explicitly in the caption.
  5. [§4.4, Eq. (9)] The notation \hat{S}^{E_{ov}}_{w_i} is used before being defined; please define the superscript E_{ov} when it is introduced.
  6. [Figure 10] The inference-speed comparison is shown without a labeled y-axis and with bars ordered inconsistently; add a quantitative axis and sort the bars by value for readability.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation; the only self-citation is non-load-bearing and the teacher dependence is an honest limitation, not a circular reduction.

full rationale

The central claim is evaluated against external ground-truth datasets (H2O, HOI4D, ARCTIC, POV-Surgery) using Chamfer distance and F-score after a global SE(3) alignment, so the target quantity is not defined in terms of EgoMono4D's own outputs or losses. The shape regularizer Lshape (Eq. 5) does align predicted per-frame point clouds to UniDepth pseudo-labels, which means the model inherits UniDepth's per-frame geometry; Appendix J openly concedes that 'the relative size of the dynamic part compared to the static part is determined by UniDepth's predictions' and that 'inherent inconsistencies in UniDepth's shape regularization' cause dynamic-part distortion and static misalignment. This is inductive-bias transfer supported by an external, previously published model, and it is acknowledged as a limitation rather than disguised as a novel prediction. It is not a circular reduction because the reported reconstruction scores are computed against real point-cloud ground truth, and the ablations (Table 2, e.g., 'w alpha = 1' degrading CD from 5.9 to 7.1) show that the outcome is sensitive to the loss design and is not forced by construction. The only self-citation, Ref. [96], is used for 3D-flow metric conventions and robot-learning motivation; the paper itself defines ADE, FDE, and precision in Appendix C, so that citation is not load-bearing. No equation defines a predicted quantity in terms of the benchmark target, no fitted parameter is renamed as a prediction, and no uniqueness claim is imported from the authors' prior work.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method rests on four main external assumptions: static-after-masking geometry for pose alignment, validity of optical flow and tracking correspondences, reliability of UniDepth pseudo-labels for shape, and coverage of EgoHOS hand-object masks. None are validated with error bounds in the paper. The free parameters are mostly standard loss and window-size hyperparameters, not fitted constants for the target result.

free parameters (4)
  • Loss weights alpha, beta, gamma, lambda, mu = 4, 5, 5, 1, 0.005
    Hand-set constants in Eq. (8). Only alpha is ablated (alpha=1 degrades performance), so the sensitivity of reported results to the other weights is not characterized.
  • Inference window size Nw = 4
    Default Nw equals the training window. Table 7 shows performance drops noticeably at Nw=2, 8 and 12, making the reported numbers dependent on this choice.
  • Overlap size No = 1
    Chosen to maximize speed; Table 7 shows No=1, 2, 3 are comparable, so this choice has minor impact.
  • Training frame sampling interval = [1,4] frames
    Randomly selected during training. Table 3 shows robustness to FPS changes only within a limited range, with degradation at very low FPS.
assumptions (4)
  • domain assumption Confidence-masked static scene assumption
    Eq. (3) solves for a single SE(3) alignment between frames after masking dynamic or occluded pixels. Any residual non-rigid motion or mask error leaks directly into camera pose estimation.
  • domain assumption Correspondence reliability assumption
    The photometric loss and pose estimation both rely on GMFlow optical flow and CoTracker long-term tracks. Section 6.4 and Appendix J admit that low FPS causes these modules to fail, which breaks the training and inference signal.
  • ad hoc to paper UniDepth pseudo-label reliability
    The shape regularization Lshape in Eq. (5) regresses model point clouds toward UniDepth predictions. Appendix J states this carries over dynamic-part size distortion and static-part misalignment from UniDepth.
  • ad hoc to paper EgoHOS mask coverage
    The pseudo-mask for photometric loss and the mask regularization Lmask use EgoHOS hand-object segmentation. Dynamic regions that EgoHOS misses are not excluded from alignment, so reconstruction quality depends on the segmentation prior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-Supervised Monocular 4D Scene Reconstruction for Egocentric Videos." pith.science (2026). https://pith.science/paper/X5AAQ6QL

@misc{pith2026241109145,
  author       = {Pith},
  title        = {Pith review of: Self-Supervised Monocular 4D Scene Reconstruction for Egocentric Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X5AAQ6QL}},
  note         = {Machine review of arXiv:2411.09145}
}
read the original abstract

Egocentric videos provide valuable insights into human interactions with the physical world, which has sparked growing interest in the computer vision and robotics communities. A critical challenge in fully understanding the geometry and dynamics of egocentric videos is dense scene reconstruction. However, the lack of high-quality labeled datasets in this field has hindered the effectiveness of current supervised learning methods. In this work, we aim to address this issue by exploring an self-supervised dynamic scene reconstruction approach. We introduce EgoMono4D, a novel model that unifies the estimation of multiple variables necessary for Egocentric Monocular 4D reconstruction, including camera intrinsic, camera poses, and video depth, all within a fast feed-forward framework. Starting from pretrained single-frame depth and intrinsic estimation model, we extend it with camera poses estimation and align multi-frame results on large-scale unlabeled egocentric videos. We evaluate EgoMono4D in both in-domain and zero-shot generalization settings, achieving superior performance in dense pointclouds sequence reconstruction compared to all baselines. EgoMono4D represents the first attempt to apply self-supervised learning for pointclouds sequence reconstruction to the label-scarce egocentric field, enabling fast, dense, and generalizable reconstruction. The interactable visualization, code and trained models are released https://egomono4d.github.io/

Figures

Figures reproduced from arXiv: 2411.09145 by the authors.

Figure 1
Figure 1. We propose EgoMono4D, a model that unifies the esti [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overview of EgoMono4D and our self-supervised training framework. The model first simultaneously predicts camera [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Visualization of dataset used for training and evaluation. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: The visualization of the dense pointcloud sequence reconstruction by EgoMono4D demonstrates its ability to effectively recover [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The visualization of the long-term 3D scene flow recovery. For clarity, we display only the pointclouds from the first and [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: More visualization of pointclouds sequence reconstruction results from EgoMono4D. [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: More visualization of long-term 3D scene flow recovery results from EgoMono4D. [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Qualitative comparison between EgoMono4D and other baseline methods. EgoMono4D demonstrates superior performance [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Visualization of original video It and the predicted procrustes-alignment confidence maps Mt and video depth Dt from EgoMono4D. CUT3R Modularized Version DS+UniDepth DUSt3R MonSt3R 0.218 0.215 0.342 1.325 5.875 Inference Speed (secs / frame) (1 Nvidia GeForce RTX3090 G…
Figure 11
Figure 11. Figure 11: Two typical failure cases of EgoMono4D arise from [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

105 extracted references · 35 canonical work pages

  1. [1]

    Map-free visual relocalization: Metric pose relative to a single im- age

    Eduardo Arnold, Jamie Wynn, Sara Vicente, Guillermo Garcia-Hernando, Aron Monszpart, Victor Prisacariu, Daniyar Turmukhambetov, and Eric Brachmann. Map-free visual relocalization: Metric pose relative to a single im- age. In European Conference on Computer Vision , pages 690–708. Springer, 2022. 2, 6, 17

    2022

  2. [2]

    Affordances from human videos as a versatile representation for robotics

    Shikhar Bahl, Russell Mendonca, Lili Chen, Unnat Jain, and Deepak Pathak. Affordances from human videos as a versatile representation for robotics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13778–13790, 2023. 3

    2023

  3. [3]

    Uncertainty-aware state space transformer for egocentric 3d hand trajectory forecasting

    Wentao Bao, Lele Chen, Libing Zeng, Zhong Li, Yi Xu, Junsong Yuan, and Yu Kong. Uncertainty-aware state space transformer for egocentric 3d hand trajectory forecasting. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13702–13711, 2023. 2, 3, 15, 16

    2023

  4. [4]

    Zoedepth: Zero-shot trans- fer by combining relative and metric depth

    Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias M ¨uller. Zoedepth: Zero-shot trans- fer by combining relative and metric depth. arXiv preprint arXiv:2302.12288, 2023. 2

    arXiv 2023

  5. [5]

    Unsuper- vised scale-consistent depth and ego-motion learning from monocular video

    Jiawang Bian, Zhichao Li, Naiyan Wang, Huangying Zhan, Chunhua Shen, Ming-Ming Cheng, and Ian Reid. Unsuper- vised scale-consistent depth and ego-motion learning from monocular video. Advances in neural information process- ing systems, 32, 2019. 2

    2019

  6. [6]

    Midas v3

    Reiner Birkl, Diana Wofk, and Matthias M ¨uller. Midas v3. 1–a model zoo for robust monocular relative depth estima- tion. arXiv preprint arXiv:2307.14460, 2023. 2, 5, 8

    arXiv 2023

  7. [7]

    Orb-slam3: An ac- curate open-source library for visual, visual–inertial, and multimap slam

    Carlos Campos, Richard Elvira, Juan J G ´omez Rodr´ıguez, Jos´e MM Montiel, and Juan D Tard ´os. Orb-slam3: An ac- curate open-source library for visual, visual–inertial, and multimap slam. IEEE Transactions on Robotics , 37(6): 1874–1890, 2021. 2

    2021

  8. [8]

    Leap-vo: Long-term effective any point tracking for visual odometry

    Weirong Chen, Le Chen, Rui Wang, and Marc Pollefeys. Leap-vo: Long-term effective any point tracking for visual odometry. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19844– 19853, 2024. 2

    2024

Show all 105 references
  1. [9]

    Vision transformer adapter for dense predictions

    Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. Vision transformer adapter for dense predictions. arXiv preprint arXiv:2205.08534, 2022. 4

    2022 arXiv

  2. [10]

    Treating mo- tion as option to reduce motion dependency in unsuper- vised video object segmentation

    Suhwan Cho, Minhyeok Lee, Seunghoon Lee, Chaewon Park, Donghyeong Kim, and Sangyoun Lee. Treating mo- tion as option to reduce motion dependency in unsuper- vised video object segmentation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pa...

    2023

  3. [11]

    Deep global registration

    Christopher Choy, Wei Dong, and Vladlen Koltun. Deep global registration. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 2514–2523, 2020. 4, 5

    2020

  4. [12]

    Dreamscene4d: Dynamic multi-object scene generation from monocular videos

    Wen-Hsuan Chu, Lei Ke, and Katerina Fragkiadaki. Dreamscene4d: Dynamic multi-object scene generation from monocular videos. arXiv preprint arXiv:2405.02280,

    arXiv

  5. [13]

    3d u-net: learn- ing dense volumetric segmentation from sparse annota- tion

    ¨Ozg¨un C ¸ ic ¸ek, Ahmed Abdulkadir, Soeren S Lienkamp, Thomas Brox, and Olaf Ronneberger. 3d u-net: learn- ing dense volumetric segmentation from sparse annota- tion. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Confer- ence, ...

    2016

  6. [14]

    Global structure-from-motion by similarity averaging

    Zhaopeng Cui and Ping Tan. Global structure-from-motion by similarity averaging. In Proceedings of the IEEE Inter- national Conference on Computer Vision , pages 864–872,

  7. [15]

    Rescaling egocentric vision: Collection, pipeline and chal- lenges for epic-kitchens-100

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Antonino Furnari, Evangelos Kazakos, Jian Ma, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, et al. Rescaling egocentric vision: Collection, pipeline and chal- lenges for epic-kitchens-100. International Journa...

    2022

  8. [16]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 14

    2010 arXiv

  9. [17]

    Mast3r-sfm: a fully-integrated solution for unconstrained structure-from-motion

    Bardienus Duisterhof, Lojze Zust, Philippe Weinzaepfel, Vincent Leroy, Yohann Cabon, and Jerome Revaud. Mast3r-sfm: a fully-integrated solution for unconstrained structure-from-motion. arXiv preprint arXiv:2409.19152 ,

    arXiv

  10. [18]

    Flowbot3d: Learning 3d articulation flow to manipulate articulated ob- jects

    Ben Eisner, Harry Zhang, and David Held. Flowbot3d: Learning 3d articulation flow to manipulate articulated ob- jects. arXiv preprint arXiv:2205.04382, 2022. 15

    2022 arXiv

  11. [19]

    Light3r-sfm: Towards feed-forward structure- from-motion

    Sven Elflein, Qunjie Zhou, S ´ergio Agostinho, and Laura Leal-Taix´e. Light3r-sfm: Towards feed-forward structure- from-motion. arXiv preprint arXiv:2501.14914, 2025. 2

    2025 arXiv

  12. [20]

    Arctic: A dataset for dexterous bimanual hand- object manipulation

    Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael J Black, and Otmar Hilliges. Arctic: A dataset for dexterous bimanual hand- object manipulation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages...

    2023

  13. [21]

    Kaolin: A pytorch library for accelerating 3d deep learning research

    Clement Fuji Tsang, Maria Shugrina, Jean Francois Lafleche, Towaki Takikawa, Jiehan Wang, Charles Loop, Wenzheng Chen, Krishna Murthy Jatavallabhula, Edward Smith, Artem Rozantsev, Or Perel, Tianchang Shen, Jun Gao, Sanja Fidler, Gavriel State, Jason Gorski, Tommy Xiang, Jiani...

    2022

  14. [22]

    First-person hand action bench- mark with rgb-d videos and 3d hand pose annotations

    Guillermo Garcia-Hernando, Shanxin Yuan, Seungryul Baek, and Tae-Kyun Kim. First-person hand action bench- mark with rgb-d videos and 3d hand pose annotations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 409–419, 2018. 3, 6, 14

    2018

  15. [23]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF Conference on Computer Vi- sio...

    2022

  16. [24]

    Deep relu networks have surprisingly few activation patterns

    Boris Hanin and David Rolnick. Deep relu networks have surprisingly few activation patterns. Advances in neural in- formation processing systems, 32, 2019. 4

    2019

  17. [25]

    Depthcrafter: Generating consistent long depth sequences for open-world videos

    Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xi- aodong Cun, Yong Zhang, Long Quan, and Ying Shan. Depthcrafter: Generating consistent long depth sequences for open-world videos. arXiv preprint arXiv:2409.02095 ,

    arXiv

  18. [26]

    Stereo4d: Learning how things move in 3d from internet stereo videos

    Linyi Jin, Richard Tucker, Zhengqi Li, David Fouhey, Noah Snavely, and Aleksander Holynski. Stereo4d: Learning how things move in 3d from internet stereo videos. arXiv preprint arXiv:2412.09621, 2024. 2, 3

    2024 arXiv

  19. [27]

    Co- tracker: It is better to track together

    Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Co- tracker: It is better to track together. arXiv preprint arXiv:2307.07635, 2023. 5, 15

    2023 arXiv

  20. [28]

    Fast encoder- based 3d from casual videos via point track processing

    Yoni Kasten, Wuyue Lu, and Haggai Maron. Fast encoder- based 3d from casual videos via point track processing. In ECCV 2024 Workshop on Wild 3D: 3D Modeling, Recon- struction, and Generation in the Wild. 2

    2024

  21. [29]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  22. [30]

    Adam: A method for stochastic opti- mization

    Diederik P Kingma. Adam: A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980, 2014. 6

    2014 arXiv

  23. [31]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4015–4026, 2023. 22

    2023

  24. [32]

    Ro- bust consistent video depth estimation

    Johannes Kopf, Xuejian Rong, and Jia-Bin Huang. Ro- bust consistent video depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1611–1621, 2021. 2

    2021

  25. [33]

    Tapvid-3d: A benchmark for tracking any point in 3d

    Skanda Koppula, Ignacio Rocco, Yi Yang, Joe Heyward, Jo˜ao Carreira, Andrew Zisserman, Gabriel Brostow, and Carl Doersch. Tapvid-3d: A benchmark for tracking any point in 3d. arXiv preprint arXiv:2407.05921, 2024. 2, 8, 15, 16

    2024 arXiv

  26. [34]

    H2o: Two hands manipulating objects for first person interaction recognition

    Taein Kwon, Bugra Tekin, Jan St ¨uhmer, Federica Bogo, and Marc Pollefeys. H2o: Two hands manipulating objects for first person interaction recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 10138–10148, 2021. 3, 6, 14

    2021

  27. [35]

    Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds

    Jiahui Lei, Yijia Weng, Adam Harley, Leonidas Guibas, and Kostas Daniilidis. Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds. arXiv preprint arXiv:2405.17421, 2024. 2

    2024 arXiv

  28. [36]

    Grounding image matching in 3d with mast3r

    Vincent Leroy, Yohann Cabon, and J ´erˆome Revaud. Grounding image matching in 3d with mast3r. arXiv preprint arXiv:2406.09756, 2024. 22

    2024 arXiv

  29. [37]

    Egocentric pre- diction of action target in 3d

    Yiming Li, Ziang Cao, Andrew Liang, Benjamin Liang, Lu- oyao Chen, Hang Zhao, and Chen Feng. Egocentric pre- diction of action target in 3d. In 2022 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages 20971–20980. IEEE, 2022. 3, 6, 14

    2022

  30. [38]

    Megasam: Accurate, fast, and robust structure and motion from casual dynamic videos

    Zhengqi Li, Richard Tucker, Forrester Cole, Qianqian Wang, Linyi Jin, Vickie Ye, Angjoo Kanazawa, Aleksander Holynski, and Noah Snavely. Megasam: Accurate, fast, and robust structure and motion from casual dynamic videos. arXiv preprint arXiv:2412.04463, 2024. 2

    2024 arXiv

  31. [39]

    Feed-forward bullet-time reconstruction of dynamic scenes from monoc- ular videos

    Hanxue Liang, Jiawei Ren, Ashkan Mirzaei, Antonio Tor- ralba, Ziwei Liu, Igor Gilitschenski, Sanja Fidler, Cen- giz Oztireli, Huan Ling, Zan Gojcic, et al. Feed-forward bullet-time reconstruction of dynamic scenes from monoc- ular videos. arXiv preprint arXiv:2412.03526, 2024. 2

    2024

  32. [40]

    Few- shot parameter-efficient fine-tuning is better and cheaper than in-context learning

    Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few- shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems, 35:1950–1965, 2022. 4

    1950

  33. [41]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023. 22

    2023 arXiv

  34. [42]

    Joint esti- mation of pose, depth, and optical flow with a competition– cooperation transformer network

    Xiaochen Liu, Tao Zhang, and Mingming Liu. Joint esti- mation of pose, depth, and optical flow with a competition– cooperation transformer network. Neural Networks, 171: 263–275, 2024. 2

    2024

  35. [43]

    Hoi4d: A 4d egocentric dataset for category-level human-object interaction

    Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, and Li Yi. Hoi4d: A 4d egocentric dataset for category-level human-object interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , p...

    2022

  36. [44]

    Robust dynamic radi- ance fields

    Yu-Lun Liu, Chen Gao, Andreas Meuleman, Hung-Yu Tseng, Ayush Saraf, Changil Kim, Yung-Yu Chuang, Jo- hannes Kopf, and Jia-Bin Huang. Robust dynamic radi- ance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13–23,

  37. [45]

    Align3r: Aligned monocu- lar depth estimation for dynamic videos

    Jiahao Lu, Tianyu Huang, Peng Li, Zhiyang Dou, Cheng Lin, Zhiming Cui, Zhen Dong, Sai-Kit Yeung, Wen- ping Wang, and Yuan Liu. Align3r: Aligned monocu- lar depth estimation for dynamic videos. arXiv preprint arXiv:2412.03079, 2024. 1, 2, 3, 5, 6, 7, 16, 17

    2024 arXiv

  38. [46]

    Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. arXiv preprint arXiv:2308.09713, 2023. 2

    2023 arXiv

  39. [47]

    Object scene flow for autonomous vehicles

    Moritz Menze and Andreas Geiger. Object scene flow for autonomous vehicles. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 3061–3070, 2015. 15

    2015

  40. [48]

    Embodiedgpt: Vision-language pre-training via embodied chain of thought

    Yao Mu, Qinglong Zhang, Mengkang Hu, Wenhai Wang, Mingyu Ding, Jun Jin, Bin Wang, Jifeng Dai, Yu Qiao, and Ping Luo. Embodiedgpt: Vision-language pre-training via embodied chain of thought. Advances in Neural Informa- tion Processing Systems, 36, 2024. 3

    2024

  41. [49]

    Orb-slam: a versatile and accurate monocular slam system

    Raul Mur-Artal, Jose Maria Martinez Montiel, and Juan D Tardos. Orb-slam: a versatile and accurate monocular slam system. IEEE transactions on robotics , 31(5):1147–1163,

  42. [50]

    Mast3r-slam: Real-time dense slam with 3d reconstruction priors

    Riku Murai, Eric Dexheimer, and Andrew J Davison. Mast3r-slam: Real-time dense slam with 3d reconstruction priors. arXiv preprint arXiv:2412.12392, 2024. 2

    2024 arXiv

  43. [51]

    R3m: A universal visual representation for robot manipulation

    Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation. arXiv preprint arXiv:2203.12601, 2022. 1, 3

    2022 arXiv

  44. [52]

    Dinov2: Learning robust visual features without supervi- sion

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervi- sion. arXiv preprint arXiv:2304.07193, 2023. 4

    2023 arXiv

  45. [53]

    From 2d to 3d: Re-thinking benchmarking of monocular depth predic- tion

    Evin Pınar ¨Ornek, Shristi Mudgal, Johanna Wald, Yida Wang, Nassir Navab, and Federico Tombari. From 2d to 3d: Re-thinking benchmarking of monocular depth predic- tion. arXiv preprint arXiv:2203.08122, 2022. 6, 14

    2022 arXiv

  46. [54]

    Aria digital twin: A new benchmark dataset for egocentric 3d machine perception

    Xiaqing Pan, Nicholas Charron, Yongqian Yang, Scott Pe- ters, Thomas Whelan, Chen Kong, Omkar Parkhi, Richard Newcombe, and Yuheng Carl Ren. Aria digital twin: A new benchmark dataset for egocentric 3d machine perception. In Proceedings of the IEEE/CVF International Conference...

    2023

  47. [55]

    Re- constructing hands in 3d with transformers

    Georgios Pavlakos, Dandan Shan, Ilija Radosavovic, Angjoo Kanazawa, David Fouhey, and Jitendra Malik. Re- constructing hands in 3d with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 9826–9836, 2024. 3

    2024

  48. [56]

    Unidepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mat- tia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition , pages 10106–10116, 2024. ...

    2024

  49. [57]

    Wilor: End-to-end 3d hand localization and reconstruction in-the-wild

    Rolandos Alexandros Potamias, Jinglei Zhang, Jiankang Deng, and Stefanos Zafeiriou. Wilor: End-to-end 3d hand localization and reconstruction in-the-wild. arXiv preprint arXiv:2409.12259, 2024. 3

    2024 arXiv

  50. [58]

    Egovlpv2: Egocentric video-language pre-training with fusion in the backbone

    Shraman Pramanick, Yale Song, Sayan Nag, Kevin Qinghong Lin, Hardik Shah, Mike Zheng Shou, Rama Chellappa, and Pengchuan Zhang. Egovlpv2: Egocentric video-language pre-training with fusion in the backbone. In Proceedings of the IEEE/CVF International Conference on Computer Vis...

    2023

  51. [59]

    Affordancellm: Grounding affordance from vision language models

    Shengyi Qian, Weifeng Chen, Min Bai, Xiong Zhou, Zhuowen Tu, and Li Erran Li. Affordancellm: Grounding affordance from vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7587–7597, 2024. 3

    2024

  52. [60]

    From one hand to multiple hands: Imitation learning for dexterous manipula- tion from single-camera teleoperation

    Yuzhe Qin, Hao Su, and Xiaolong Wang. From one hand to multiple hands: Imitation learning for dexterous manipula- tion from single-camera teleoperation. IEEE Robotics and Automation Letters, 7(4):10873–10881, 2022. 3

    2022

  53. [61]

    Deep learning- based depth estimation methods from monocular image and videos: A comprehensive survey.ACM Computing Surveys,

    Uchitha Rajapaksha, Ferdous Sohel, Hamid Laga, Dean Diepeveen, and Mohammed Bennamoun. Deep learning- based depth estimation methods from monocular image and videos: A comprehensive survey.ACM Computing Surveys,

  54. [62]

    Nerf- slam: Real-time dense monocular slam with neural radi- ance fields

    Antoni Rosinol, John J Leonard, and Luca Carlone. Nerf- slam: Real-time dense monocular slam with neural radi- ance fields. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 3437–3444. IEEE, 2023. 2

    2023

  55. [63]

    Identification and cor- rection of flying pixels in range camera data

    Alexander Sabov and J ¨org Kr ¨uger. Identification and cor- rection of flying pixels in range camera data. In Proceed- ings of the 24th Spring Conference on Computer Graphics, pages 135–142, 2008. 5, 15

    2008

  56. [64]

    Structure-from-motion revisited

    Johannes L Schonberger and Jan-Michael Frahm. Structure-from-motion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4104–4113, 2016. 1, 2

    2016

  57. [65]

    Pixelwise view selection for unstructured multi-view stereo

    Johannes L Sch ¨onberger, Enliang Zheng, Jan-Michael Frahm, and Marc Pollefeys. Pixelwise view selection for unstructured multi-view stereo. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Nether- lands, October 11-14, 2016, Proceedings, Part III 14, pag...

    2016

  58. [66]

    Toolflownet: Robotic manipulation with tools via predicting tool flow from point clouds

    Daniel Seita, Yufei Wang, Sarthak J Shetty, Edward Yao Li, Zackory Erickson, and David Held. Toolflownet: Robotic manipulation with tools via predicting tool flow from point clouds. In Conference on Robot Learning , pages 1038–

  59. [67]

    Understanding human hands in contact at inter- net scale

    Dandan Shan, Jiaqi Geng, Michelle Shu, and David F Fouhey. Understanding human hands in contact at inter- net scale. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9869–9878,

  60. [68]

    Motion-based object segmentation based on dense rgb-d scene flow

    Lin Shao, Parth Shah, Vikranth Dwaracherla, and Jeannette Bohg. Motion-based object segmentation based on dense rgb-d scene flow. IEEE Robotics and Automation Letters, 3 (4):3797–3804, 2018. 15

    2018

  61. [69]

    Swindepth: Unsupervised depth estimation using monocular sequences via swin trans- former and densely cascaded network

    Dongseok Shim and H Jin Kim. Swindepth: Unsupervised depth estimation using monocular sequences via swin trans- former and densely cascaded network. In 2023 IEEE Inter- national Conference on Robotics and Automation (ICRA) , pages 4983–4990. IEEE, 2023. 2

    2023

  62. [70]

    Flowcam: training generalizable 3d radiance fields without camera poses via pixel-aligned scene flow

    Cameron Smith, Yilun Du, Ayush Tewari, and Vincent Sitz- mann. Flowcam: training generalizable 3d radiance fields without camera poses via pixel-aligned scene flow. arXiv preprint arXiv:2306.00180, 2023. 4, 5

    2023 arXiv

  63. [71]

    Flowmap: High-quality camera poses, in- trinsics, and depth via gradient descent

    Cameron Smith, David Charatan, Ayush Tewari, and Vin- cent Sitzmann. Flowmap: High-quality camera poses, in- trinsics, and depth via gradient descent. arXiv preprint arXiv:2404.15259, 2024. 2, 3, 4, 5, 6, 8

    2024 arXiv

  64. [72]

    Kick back & relax: Learning to reconstruct the world by watching slowtv

    Jaime Spencer, Chris Russell, Simon Hadfield, and Richard Bowden. Kick back & relax: Learning to reconstruct the world by watching slowtv. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 15768–15779, 2023. 2, 3, 5, 6, 18

    2023

  65. [73]

    Kick back & relax++: Scaling beyond ground- truth depth with slowtv & cribstv

    Jaime Spencer, Chris Russell, Simon Hadfield, and Richard Bowden. Kick back & relax++: Scaling beyond ground- truth depth with slowtv & cribstv. arXiv preprint arXiv:2403.01569, 2024. 2, 5, 6, 18

    2024 arXiv

  66. [74]

    A unified transformer frame- work for group-based segmentation: Co-segmentation, co- saliency detection and video salient object detection

    Yukun Su, Jingliang Deng, Ruizhou Sun, Guosheng Lin, Hanjing Su, and Qingyao Wu. A unified transformer frame- work for group-based segmentation: Co-segmentation, co- saliency detection and video salient object detection. IEEE Transactions on Multimedia, 26:313–325, 2023. 22

    2023

  67. [75]

    3dgstream: On-the-fly training of 3d gaussians for efficient streaming of photo-realistic free- viewpoint videos

    Jiakai Sun, Han Jiao, Guangyuan Li, Zhanjie Zhang, Lei Zhao, and Wei Xing. 3dgstream: On-the-fly training of 3d gaussians for efficient streaming of photo-realistic free- viewpoint videos. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, ...

    2024

  68. [76]

    Dynamo-depth: fix- ing unsupervised depth estimation for dynamical scenes

    Yihong Sun and Bharath Hariharan. Dynamo-depth: fix- ing unsupervised depth estimation for dynamical scenes. Advances in Neural Information Processing Systems , 36,

  69. [77]

    Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras

    Zachary Teed and Jia Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. Advances in neural information processing systems , 34:16558–16569,

  70. [78]

    Cutting edge, flagship camera with intelli- gent feedback and resolution

    Vicon Vantage. Cutting edge, flagship camera with intelli- gent feedback and resolution. 6

  71. [79]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 4, 14

    2017

  72. [80]

    Three-dimensional scene flow

    Sundar Vedula, Simon Baker, Peter Rander, Robert Collins, and Takeo Kanade. Three-dimensional scene flow. In Pro- ceedings of the Seventh IEEE International Conference on Computer Vision, pages 722–729. IEEE, 1999. 8, 15

    1999

  73. [81]

    3d reconstruction with spatial memory

    Hengyi Wang and Lourdes Agapito. 3d reconstruction with spatial memory. arXiv preprint arXiv:2408.16061, 2024. 2, 3

    2024 arXiv

  74. [82]

    Vggsfm: Visual geometry grounded deep structure from motion

    Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny. Vggsfm: Visual geometry grounded deep structure from motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 21686–21697, 2024. 2, 22

    2024

  75. [83]

    Shape of motion: 4d reconstruction from a single video

    Qianqian Wang, Vickie Ye, Hang Gao, Jake Austin, Zhengqi Li, and Angjoo Kanazawa. Shape of motion: 4d reconstruction from a single video. arXiv preprint arXiv:2407.13764, 2024. 2, 16

    2024

  76. [84]

    Continuous 3d perception model with persistent state

    Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state. arXiv preprint arXiv:2501.12387, 2025. 1, 2, 3, 6, 7, 16, 17

    2025 arXiv

  77. [85]

    Pov-surgery: A dataset for egocentric hand and tool pose estimation during surgi- cal activities

    Rui Wang, Sophokles Ktistakis, Siwei Zhang, Mirko Meboldt, and Quentin Lohmeyer. Pov-surgery: A dataset for egocentric hand and tool pose estimation during surgi- cal activities. In International Conference on Medical Im- age Computing and Computer-Assisted Intervention, pages...

    2023

  78. [86]

    Dust3r: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 20697–20709, 2024. 1, 2, 3, 6, 7, 16, 17, 22

    2024

  79. [87]

    Tar- tanvo: A generalizable learning-based vo

    Wenshan Wang, Yaoyu Hu, and Sebastian Scherer. Tar- tanvo: A generalizable learning-based vo. In Conference on Robot Learning, pages 1761–1772. PMLR, 2021. 2

    2021

  80. [88]

    Neural video depth stabilizer

    Yiran Wang, Min Shi, Jiaqi Li, Zihao Huang, Zhiguo Cao, Jianming Zhang, Ke Xian, and Guosheng Lin. Neural video depth stabilizer. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision , pages 9466–9476,

  81. [89]

    Egocentric video comprehension via large language model inner speech

    Ying Wang, Dongdong Sun, Rui Chen, Yanlai Yang, and Mengye Ren. Egocentric video comprehension via large language model inner speech. In 3rd International Ego4D Workshop, 2023. 3

    2023

  82. [90]

    Any-point trajectory modeling for policy learning

    Chuan Wen, Xingyu Lin, John So, Kai Chen, Qi Dou, Yang Gao, and Pieter Abbeel. Any-point trajectory modeling for policy learning. arXiv preprint arXiv:2401.00025, 2023. 1, 3

    2023 arXiv

  83. [91]

    Moving object segmentation: All you need is sam (and flow)

    Junyu Xie, Charig Yang, Weidi Xie, and Andrew Zisser- man. Moving object segmentation: All you need is sam (and flow). arXiv preprint arXiv:2404.12389, 2024. 22

    2024 arXiv

  84. [92]

    Gmflow: Learning optical flow via global matching

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. Gmflow: Learning optical flow via global matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8121–8130,

  85. [93]

    Depth anything: Un- leashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Ji- ashi Feng, and Hengshuang Zhao. Depth anything: Un- leashing the power of large-scale unlabeled data. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10371–10381, 2024. 2

    2024

  86. [94]

    Every pixel counts: Unsupervised geome- try learning with holistic 3d motion understanding

    Zhenheng Yang, Peng Wang, Yang Wang, Wei Xu, and Ram Nevatia. Every pixel counts: Unsupervised geome- try learning with holistic 3d motion understanding. In Pro- ceedings of the European conference on computer vision (ECCV) workshops, pages 0–0, 2018. 2, 5, 6

    2018

  87. [95]

    Met- ric3d: Towards zero-shot metric 3d prediction from a sin- gle image

    Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Met- ric3d: Towards zero-shot metric 3d prediction from a sin- gle image. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 9043–9053, 2023. 1, 2

    2023

  88. [96]

    General flow as foundation affordance for scalable robot learning

    Chengbo Yuan, Chuan Wen, Tong Zhang, and Yang Gao. General flow as foundation affordance for scalable robot learning. arXiv preprint arXiv:2401.11439, 2024. 1, 2, 3, 8, 15, 16

    2024 arXiv

  89. [97]

    Recent trends in 3d reconstruction of general non-rigid scenes

    Raza Yunus, Jan Eric Lenssen, Michael Niemeyer, Yiyi Liao, Christian Rupprecht, Christian Theobalt, Gerard Pons-Moll, Jia-Bin Huang, Vladislav Golyanik, and Eddy Ilg. Recent trends in 3d reconstruction of general non-rigid scenes. In Computer Graphics Forum, page e15062. Wiley...

    2024

  90. [98]

    Egogaussian: Dynamic scene understanding from egocentric video with 3d gaussian splatting

    Daiwei Zhang, Gengyan Li, Jiajie Li, Micka ¨el Bressieux, Otmar Hilliges, Marc Pollefeys, Luc Van Gool, and Xi Wang. Egogaussian: Dynamic scene understanding from egocentric video with 3d gaussian splatting. arXiv preprint arXiv:2406.19811, 2024. 2, 3

    2024 arXiv

  91. [99]

    Monst3r: A simple approach for esti- mating geometry in the presence of motion

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jam- pani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. Monst3r: A simple approach for esti- mating geometry in the presence of motion. arXiv preprint arXiv:2410.03825, 2024. 1, 2, 3, 5, 6, 7, 16, 17, 20, 21, 22

    arXiv

  92. [100]

    Fine-grained egocentric hand-object segmentation: Dataset, model, and applications

    Lingzhi Zhang, Shenghao Zhou, Simon Stent, and Jianbo Shi. Fine-grained egocentric hand-object segmentation: Dataset, model, and applications. In European Conference on Computer Vision, pages 127–145. Springer, 2022. 1, 2, 3, 5, 6

    2022

  93. [101]

    Go-slam: Global optimization for consistent 3d instant reconstruction

    Youmin Zhang, Fabio Tosi, Stefano Mattoccia, and Mat- teo Poggi. Go-slam: Global optimization for consistent 3d instant reconstruction. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 3727– 3737, 2023. 2

    2023

  94. [102]

    Structure and motion from casual videos

    Zhoutong Zhang, Forrester Cole, Zhengqi Li, Michael Ru- binstein, Noah Snavely, and William T Freeman. Structure and motion from casual videos. In European Conference on Computer Vision, pages 20–37. Springer, 2022. 2

    2022

  95. [103]

    Particlesfm: Exploiting dense point tra- jectories for localizing moving cameras in the wild

    Wang Zhao, Shaohui Liu, Hengkai Guo, Wenping Wang, and Yong-Jin Liu. Particlesfm: Exploiting dense point tra- jectories for localizing moving cameras in the wild. In Eu- ropean Conference on Computer Vision , pages 523–542. Springer, 2022. 2

    2022

  96. [104]

    Pointodyssey: A large-scale synthetic dataset for long-term point tracking

    Yang Zheng, Adam W Harley, Bokui Shen, Gordon Wet- zstein, and Leonidas J Guibas. Pointodyssey: A large-scale synthetic dataset for long-term point tracking. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 19855–19865, 2023. 22 Appendix Th...

    2023

  97. [2021]

    1, 2, 6, 7, 16, 17, 20, 21

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.