REVIEW 3 major objections 4 minor 87 references
Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Space
T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Common3D learns 3D morphable models of everyday objects from casual videos alone, with no manual 3D annotations.
desk verdict Solid self-supervised 3DMM paper with a real but overstated 'completely self-supervised' claim — the head-to-head numbers are entangled with the authors' own UOP3D poses, but the method is coherent, well-ablated, and deserves a serious review. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the coupling between the deformable template mesh and a contrastive appearance objective. The template supplies a shared 3D coordinate frame: after deformation, corresponding vertices across instances are known by construction, and those correspondences are used to supervise the feature adapter. The appearance loss treats each 2D image feature as a query over all vertex features plus a background token, computing a surface-probability distribution via a softmax with temperature $\kappa$, while the target distribution is the Euclidean-distance surface probability on the mesh. Minimizing their cross-entropy forces the 2D features and the 3D vertex features to agree on which point of the object a pixel belongs to, with farthest-point vertex sampling and the background token preventing collapse to trivial constant features. The deformed mesh vertices also give direct point access for chamfer-distance shape fitting and rasterization-based mask fitting, so geometry and appearance are optimized jointly in the same canonical frame.
What would settle it
Train the same recipe on a category with strong mirror or rotational symmetry, such as mugs or bottles, and compare the learned template and predicted poses against independent structure-from-motion reconstructions after reflecting one symmetry-related reconstruction; if predictions flip along the symmetry axis instead of staying consistent, the canonical pose initialization rather than the contrastive correspondence mechanism is carrying the result.
Extended reading notes
Core claim
Common3D's central claim is that a deformable 3D morphable model can be trained end-to-end without any 3D supervision and then serve as one prior that solves multiple single-image tasks. The model is a learned category-level template mesh, extracted from a signed distance field via differentiable marching tetrahedra, whose vertices carry semantic feature vectors generated by a feature field. An image-conditioned affine deformation field morphs the template to the instance geometry, and a frozen DINOv2 backbone with a small trainable adapter provides image features. Training combines geometric losses, silhouette overlap, distance-transform expansion, chamfer distance to per-video structure-from-motion point clouds, smoothness and Eikonal regularization, with a contrastive appearance loss in which every image feature must be explained by a vertex feature through surface probabilities derived from the current mesh. The paper argues this makes the adapter learn correspondence features that are more distinct than raw DINOv2 features, and makes inverse rendering on those features reliably recover pose. On this basis, Common3D claims to be the first completely self-supervised method that estimates shape, pose, and semantic correspondence of common objects from a single image.
Load-bearing premise
The entire model is trained in a canonical coordinate frame whose camera poses come from an earlier unsupervised pose-estimation method (UOP3D [52]); if those poses are systematically biased, the learned template, deformations, features, and all downstream evaluations inherit that bias, and the comparison against UOP3D is not fully independent.
Editorial extensions
If this is right
- A single category model trained on up to 50 videos per category yields zero-shot 3D pose, instance segmentation, and semantic correspondence on unseen in-the-wild images of that category.
- The average gains of 6.3 and 4.4 points in 30-degree pose accuracy on PASCAL3D+ and ObjectNet3D imply that the correspondence-aware features transfer across the domain gap from CO3D videos to images in the wild.
- Categories with little shape variation across instances, such as cellphone, microwave, and toaster, do not improve, suggesting the deformation model contributes most where instance geometry varies.
- Because the adapter is trained jointly with the geometry, correspondence quality can improve as the shape model improves, enabling a self-improving loop between 2D features and the 3D prior.
- The same trained model supports multiple tasks, so task-specific heads or per-task training data are not required at test time.
Reading between the lines
- A natural extension is to test whether the same framework scales to categories with large topological variation, since DMTet and the learned SDF should in principle handle topology changes that rigid template meshes cannot.
- The reliance on a separately estimated canonical pose suggests a testable variant: replace the UOP3D pose initialization with poses from multi-view triangulation or object symmetry and compare downstream accuracy, isolating how much of the gain comes from the feature-adapter mechanism.
- If correspondence features genuinely improve through this loop, the trained adapter could transfer to related tasks such as part segmentation, keypoint discovery, or video object tracking, where the same pixel-to-surface question is asked.
- The small training budget, under ten hours per category on a single GPU, raises the possibility of interactive model creation: a user shoots a short video of an object and immediately obtains a category-specific 3D morphable model.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Common3D, a method for learning category-specific 3D morphable models (3DMMs) from collections of object-centric videos without manual 3D annotations. The model combines a DMTet-based hybrid volumetric-mesh template, an instance-level affine deformation field conditioned on an image latent code, and a neural feature field on the mesh surface. Appearance is represented by DINOv2 features refined by a learned adapter, and training uses geometric losses (mask, chamfer, SDF, deformation regularization) together with a contrastive appearance objective based on surface probabilities. At inference, pose is estimated by inverse rendering in feature space and shape by the latent encoder. The paper evaluates single-image 3D pose, instance segmentation, and semantic correspondence on PASCAL3D+, ObjectNet3D, and SPair-71k, reporting improvements over the unsupervised baseline UOP3D and claims to be the first completely self-supervised method to solve these tasks jointly. Code is released.
Significance. If the central claims are sustained, this is a meaningful contribution: it extends 3DMMs beyond faces and bodies to common object categories without manual 3D supervision, demonstrates that feature-space appearance representations can make inverse rendering tractable, and shows improvements on external benchmarks. The evaluation is conducted on held-out in-the-wild datasets, so the main results are not fitted to the test sets, and the ablation study in Table 6 supports the importance of the adapter and the dense appearance loss. The release of code is a further strength. However, the headline 'completely self-supervised' claim is partially undermined by the use of PointRend masks, precomputed structure-from-motion point clouds, and UOP3D-derived canonical camera poses, and the comparison against UOP3D is entangled with the training coordinate frame. The contribution would be strengthened substantially by an independent canonicalization experiment and by uncertainty quantification on the aggregate metrics.
major comments (3)
- [Sec. 3.3, Sec. 4.1, Tables 1-2] The canonical camera poses used to train Common3D are obtained by adopting UOP3D [52], as stated in Sec. 3.3 and Sec. 4.1. The template shape, the deformation model, and the contrastive appearance objective (Eqs. 11-12) are all optimized in the coordinate frame defined by those poses, and inference in Eq. (13) is performed against features learned in that same frame. Consequently, the reported pose improvements over UOP3D (69.2% to 75.3% on PASCAL3D+ and 52.4% to 56.8% on ObjectNet3D) may reflect a better optimization or smoother correspondence features operating on UOP3D's pseudo-labels rather than an independent category-level 3D prior. The paper should either retrain with an independent canonicalization, such as a different unsupervised pose aligner or a canonical frame derived directly from the SFM point clouds, or demonstrate robustness to the choice of canonical frame. Without this, the comparison is closer to a teacher-student evaluation than a head-to-head one, and the 'completely self-supervised' claim is overstated.
- [Abstract, Sec. 1, Conclusion] The claim that Common3D is 'completely self-supervised' is contradicted by the training data dependencies described in Sec. 3.3 and Sec. 4.1: masks come from PointRend [26] in Eq. (2), a supervised instance segmentation method; 3D point clouds come from a precomputed SFM pipeline [48] in Eq. (4); and canonical camera poses come from UOP3D [52] in Eqs. (11)-(12). While these are not manual 3D shape or pose annotations, they are precomputed geometric pseudo-labels that shape the learned model. Please qualify the self-supervision claim at the outset, for example by stating explicitly that the method uses no manual keypoint, pose, or 3D shape annotations but does rely on precomputed masks, SFM reconstructions, and an unsupervised canonicalization from a prior method.
- [Tables 1-5, Sec. 4.2] The paper reports only point estimates for the main quantitative results, yet the text states that the improvements are 'significant' (Sec. 4.2). Given the per-category variance visible in Tables 1-2 (e.g., ObjectNet3D suitcase: UOP3D 44.7 vs. Ours 38.5; keyboard: 26.9 vs. 34.5) and the modest per-category sample sizes, the aggregate gains need uncertainty quantification, such as confidence intervals from bootstrapping over test images or error bars across training seeds. At minimum, this would make the headline claims about the average improvement over UOP3D interpretable.
minor comments (4)
- [Eq. (9)] The denominator of the surface probability in Eq. (9) appears to be missing a negative sign in the exponent: it should be exp(-||vi - vk||^2 / (2 sigma^2)) to match the numerator and the usual Gaussian form.
- [Sec. 4.1] The sentence 'We achieve adopt the method proposed in [52]...' contains a grammatical error; it should read 'We adopt the method proposed in [52]...' or 'We achieve this by adopting...'.
- [Sec. 4.4, Table 5] The PCK@0.1 metric is used in Table 5 and Sec. 4.4 without a definition, and the fixed weighting 0.8/0.2 between DINOv2 and adapter features is reported without justification or ablation. Please define the metric and either ablate or justify the weighting.
- [Sec. 3.3] Farthest point sampling is mentioned as part of the appearance objective, but the main text defers the details; the supplementary Table 7 lists 150 sampled vertices. Please state this number and the purpose of the sampling directly in Sec. 3.3 for readability.
Circularity Check
No definitional circularity: the 3DMM, deformations, and contrastive features are trained from independent objectives and evaluated on external benchmarks; the only entanglement is adoption of the authors' own UOP3D for canonical poses, which is a benchmark-independence caveat rather than a circular reduction.
full rationale
The derivation chain is self-contained with respect to the main claims. The category template, instance deformation field, feature field, and adapter are optimized by geometric losses (Eqs. 2-7), chamfer and SDF regularization, and a contrastive appearance loss (Eqs. 8-10) whose supervision is the model's own mesh correspondence probabilities (Eq. 9) - a self-supervised training signal, not a test-set fit. Pose at inference is estimated by inverse rendering (Eq. 13) and evaluated on held-out PASCAL3D+ and ObjectNet3D, so the reported 30-degree accuracies are not fitted to the evaluation data. The principal caveat is in Sec. 3.3: 'We achieve this by adopting the method proposed in [52] to get initial camera pose annotations in an unsupervised manner.' Reference [52] (UOP3D) shares two authors with this paper and is also the primary baseline; thus the canonical coordinate frame in which all geometry, deformations, and correspondences are learned is inherited from the baseline, and the comparison is partly a same-family comparison. This weakens the independence of the +6.3/+4.4 pose improvements and the 'completely self-supervised' wording, but it is not a case where a prediction reduces by construction to its inputs: the model still learns a deformable template and feature adapter that can produce poses different from UOP3D. No circular step of the defined kinds can be exhibited, so the score reflects only the minor self-citation and baseline dependence.
Assumptions & free parameters
free parameters (4)
- Loss weights =
lambda_app=0.1, lambda_CD=0.1, lambda_m=1.0, lambda_mdt=100, lambda_sdf=0.01, lambda_def=0.1, lambda_def-sm=0.01
- Contrastive temperature kappa =
14.3
- Tetrahedral grid size =
16
- Vertex sampling subset size =
150
assumptions (5)
- domain assumption Canonical camera poses estimated by UOP3D [52] are accurate enough to supervise the joint training.
- domain assumption Frozen DINOv2 features provide a shared semantic embedding across instances of a category that an adapter can refine into dense 2D-3D correspondences.
- domain assumption CO3D object masks from PointRend and SfM point clouds are sufficiently accurate for the mask, distance-transform, and chamfer losses.
- standard math DMTet yields a watertight mesh with usable gradients, and the Eikonal regularization keeps the SDF valid away from the surface.
- domain assumption An affine deformation field per instance plus a shared template captures the shape variation of each category.
invented entities (1)
-
Background feature beta
Cite this review
Pith. "Pith review of Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Space." pith.science (2026). https://pith.science/paper/LCW56JG4
@misc{pith2026250421749,
author = {Pith},
title = {Pith review of: Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Space},
year = {2026},
howpublished = {\url{https://pith.science/paper/LCW56JG4}},
note = {Machine review of arXiv:2504.21749}
}
read the original abstract
3D morphable models (3DMMs) are a powerful tool to represent the possible shapes and appearances of an object category. Given a single test image, 3DMMs can be used to solve various tasks, such as predicting the 3D shape, pose, semantic correspondence, and instance segmentation of an object. Unfortunately, 3DMMs are only available for very few object categories that are of particular interest, like faces or human bodies, as they require a demanding 3D data acquisition and category-specific training process. In contrast, we introduce a new method, Common3D, that learns 3DMMs of common objects in a fully self-supervised manner from a collection of object-centric videos. For this purpose, our model represents objects as a learned 3D template mesh and a deformation field that is parameterized as an image-conditioned neural network. Different from prior works, Common3D represents the object appearance with neural features instead of RGB colors, which enables the learning of more generalizable representations through an abstraction from pixel intensities. Importantly, we train the appearance features using a contrastive objective by exploiting the correspondences defined through the deformable template mesh. This leads to higher quality correspondence features compared to related works and a significantly improved model performance at estimating 3D object pose and semantic correspondence. Common3D is the first completely self-supervised method that can solve various vision tasks in a zero-shot manner.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[52]
Unsupervised learning of category-level 3d pose 10 from object-centric videos
Leonhard Sommer, Artur Jesslen, Eddy Ilg, and Adam Ko- rtylewski. Unsupervised learning of category-level 3d pose 10 from object-centric videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 22787–22796, 2024. 2, 5, 6, 7, 8, 3
work page 2024
-
[26]
Pointrend: Image segmentation as rendering
Alexander Kirillov, Yuxin Wu, Kaiming He, and Ross Gir- shick. Pointrend: Image segmentation as rendering. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9799–9808, 2020. 5, 6 9
work page 2020
-
[48]
Structure-from-Motion Revisited
Johannes Lutz Sch ¨onberger and Jan-Michael Frahm. Structure-from-Motion Revisited. In Conference on Com- puter Vision and Pattern Recognition (CVPR), 2016. 5, 6
work page 2016
-
[1]
Inverse rendering of faces with a 3d morphable model
Oswald Aldrian and William AP Smith. Inverse rendering of faces with a 3d morphable model. IEEE transactions on pattern analysis and machine intelligence, 35(5):1080–1093,
-
[2]
An image is worth 16x16 words: Trans- formers for image recognition at scale
Dosovitskiy Alexey. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv: 2010.11929, 2020. 7
arXiv 2010
-
[3]
Scape: shape completion and animation of people
Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Se- bastian Thrun, Jim Rodgers, and James Davis. Scape: shape completion and animation of people. In ACM SIGGRAPH 2005 Papers, pages 408–416. 2005. 2
2005
-
[4]
Coke: Contrastive learning for robust keypoint detec- tion
Yutong Bai, Angtian Wang, Adam Kortylewski, and Alan Yuille. Coke: Contrastive learning for robust keypoint detec- tion. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 65–74, 2023. 5
2023
-
[5]
A morphable model for the synthesis of 3d faces
V BLANZ and T VETTER. A morphable model for the synthesis of 3d faces. In Computer graphics proceedings, annual conference series , pages 187–194. Association for Computing Machinery SIGGRAPH, 1999. 1, 2
1999
Show all 87 references
-
[6]
Emerg- ing properties in self-supervised vision transformers
Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 4
2021
-
[7]
Shapenet: An information-rich 3d model repository
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015. 3
2015 arXiv
-
[8]
Learning implicit fields for generative shape modeling
Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 5939–5948, 2019. 2
2019
-
[9]
Neural parametric gaussians for monocular non-rigid object reconstruction
Devikalyan Das, Christopher Wewer, Raza Yunus, Eddy Ilg, and Jan Eric Lenssen. Neural parametric gaussians for monocular non-rigid object reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 10715–10725, 2024. 3
2024
-
[10]
Deformed implicit field: Modeling 3d shapes with learned dense correspon- dence
Yu Deng, Jiaolong Yang, and Xin Tong. Deformed implicit field: Modeling 3d shapes with learned dense correspon- dence. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 10286–10296,
-
[11]
Occlusion-aware 3d morphable models and an illumination prior for face image analysis
Bernhard Egger, Sandro Sch ¨onborn, Andreas Schnei- der, Adam Kortylewski, Andreas Morel-Forster, Clemens Blumer, and Thomas Vetter. Occlusion-aware 3d morphable models and an illumination prior for face image analysis. International Journal of Computer Vision , 126:1269–1287,
-
[12]
Shape and viewpoint without keypoints
Shubham Goel, Angjoo Kanazawa, and Jitendra Malik. Shape and viewpoint without keypoints. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XV 16 , pages 88–104. Springer, 2020. 3
2020
-
[13]
Dif- ferentiable stereopsis: Meshes from multiple views using differentiable rendering
Shubham Goel, Georgia Gkioxari, and Jitendra Malik. Dif- ferentiable stereopsis: Meshes from multiple views using differentiable rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8635–8644, 2022. 4
2022
-
[14]
Zero-shot category-level object pose estimation
Walter Goodwin, Sagar Vaze, Ioannis Havoutis, and Ingmar Posner. Zero-shot category-level object pose estimation. In European Conference on Computer Vision, pages 516–532. Springer, 2022. 7, 2
2022
-
[15]
Implicit geometric regularization for learning shapes
Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometric regularization for learning shapes. arXiv preprint arXiv:2002.10099, 2020. 5
2002 arXiv
-
[16]
Atlasnet: a papier- mˆach´e approach to learning 3d surface generation (2018)
Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. Atlasnet: a papier- mˆach´e approach to learning 3d surface generation (2018). arXiv preprint arXiv:1802.05384, 11, 1802. 3
2018 arXiv
-
[17]
Leveraging 2d data to learn textured 3d mesh generation
Paul Henderson, Vagia Tsiminaki, and Christoph H Lampert. Leveraging 2d data to learn textured 3d mesh generation. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 7498–7507, 2020. 3
2020
-
[18]
Self-supervised 3d mesh reconstruction from single images
Tao Hu, Liwei Wang, Xiaogang Xu, Shu Liu, and Jiaya Jia. Self-supervised 3d mesh reconstruction from single images. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 6002–6011, 2021. 3
2021
-
[19]
Scops: Self-supervised co-part segmentation
Wei-Chih Hung, Varun Jampani, Sifei Liu, Pavlo Molchanov, Ming-Hsuan Yang, and Jan Kautz. Scops: Self-supervised co-part segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 869–878, 2019. 3
2019
-
[20]
Farm3d: Learning articu- lated 3d animals by distilling 2d diffusion
Tomas Jakab, Ruining Li, Shangzhe Wu, Christian Rup- precht, and Andrea Vedaldi. Farm3d: Learning articu- lated 3d animals by distilling 2d diffusion. In 2024 Inter- national Conference on 3D Vision (3DV) , pages 852–861. IEEE, 2024. 3
2024
-
[21]
Total cap- ture: A 3d deformation model for tracking faces, hands, and bodies
Hanbyul Joo, Tomas Simon, and Yaser Sheikh. Total cap- ture: A 3d deformation model for tracking faces, hands, and bodies. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8320–8329, 2018. 2
2018
-
[22]
Learning category-specific mesh reconstruc- tion from image collections
Angjoo Kanazawa, Shubham Tulsiani, Alexei A Efros, and Jitendra Malik. Learning category-specific mesh reconstruc- tion from image collections. In Proceedings of the Euro- pean Conference on Computer Vision (ECCV) , pages 371– 386, 2018. 3
2018
-
[23]
Learning category-specific mesh reconstruc- tion from image collections
Angjoo Kanazawa, Shubham Tulsiani, Alexei A Efros, and Jitendra Malik. Learning category-specific mesh reconstruc- tion from image collections. In Proceedings of the Euro- pean Conference on Computer Vision (ECCV) , pages 371– 386, 2018. 2
2018
-
[24]
Category-specific object reconstruction from a single image
Abhishek Kar, Shubham Tulsiani, Joao Carreira, and Jiten- dra Malik. Category-specific object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1966–1974,
1966
-
[25]
Adam: A method for stochastic opti- mization
Diederik P Kingma. Adam: A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980, 2014. 7
2014 arXiv
-
[27]
To the point: Correspondence-driven monocular 3d category reconstruc- tion
Filippos Kokkinos and Iasonas Kokkinos. To the point: Correspondence-driven monocular 3d category reconstruc- tion. Advances in Neural Information Processing Systems , 34:7760–7772, 2021. 3
2021
-
[28]
Articulation-aware canonical surface map- ping
Nilesh Kulkarni, Abhinav Gupta, David F Fouhey, and Shub- ham Tulsiani. Articulation-aware canonical surface map- ping. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 452–461, 2020. 3
2020
-
[29]
Modular primitives for high-performance differentiable rendering
Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. Modular primitives for high-performance differentiable rendering. ACM Transac- tions on Graphics, 39(6), 2020. 7
2020
-
[30]
The visual hull concept for silhouette-based image understanding
Aldo Laurentini. The visual hull concept for silhouette-based image understanding. IEEE Transactions on pattern analysis and machine intelligence, 16(2):150–162, 1994. 5
1994
-
[31]
Self-supervised single-view 3d reconstruction via semantic consistency
Xueting Li, Sifei Liu, Kihwan Kim, Shalini De Mello, Varun Jampani, Ming-Hsuan Yang, and Jan Kautz. Self-supervised single-view 3d reconstruction via semantic consistency. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Par...
2020
-
[32]
Learning the 3d fauna of the web
Zizhang Li, Dor Litvak, Ruining Li, Yunzhi Zhang, Tomas Jakab, Christian Rupprecht, Shangzhe Wu, Andrea Vedaldi, and Jiajun Wu. Learning the 3d fauna of the web. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9752–9762, 2024. 2, 3
2024
-
[33]
Markerless motion cap- ture of multiple characters using multiview image segmenta- tion
Yebin Liu, Juergen Gall, Carsten Stoll, Qionghai Dai, Hans- Peter Seidel, and Christian Theobalt. Markerless motion cap- ture of multiple characters using multiview image segmenta- tion. IEEE transactions on pattern analysis and machine intelligence, 35(11):2720–2735, 2013. 2
2013
-
[34]
Smpl: a skinned multi- person linear model
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. Smpl: a skinned multi- person linear model. ACM Transactions on Graphics (TOG), 34(6):1–16, 2015. 2
2015
-
[35]
Im- proving semantic correspondence with viewpoint-guided spherical maps
Octave Mariotti, Oisin Mac Aodha, and Hakan Bilen. Im- proving semantic correspondence with viewpoint-guided spherical maps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19521– 19530, 2024. 2, 8
2024
-
[36]
Occupancy networks: Learning 3d reconstruction in function space
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Se- bastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4460–4470, 2019. 2
2019
-
[37]
Spair-71k: A large-scale benchmark for semantic correspon- dence
Juhong Min, Jongmin Lee, Jean Ponce, and Minsu Cho. Spair-71k: A large-scale benchmark for semantic correspon- dence. arXiv preprint arXiv:1908.10543, 2019. 2, 6, 8
1908 arXiv
-
[38]
Share with thy neighbors: Single-view reconstruction by cross-instance consistency
Tom Monnier, Matthew Fisher, Alexei A Efros, and Mathieu Aubry. Share with thy neighbors: Single-view reconstruction by cross-instance consistency. In European Conference on Computer Vision, pages 285–303. Springer, 2022. 3
2022
-
[39]
Continu- ous surface embeddings
Natalia Neverova, David Novotny, Marc Szafraniec, Vasil Khalidov, Patrick Labatut, and Andrea Vedaldi. Continu- ous surface embeddings. Advances in Neural Information Processing Systems, 33:17258–17270, 2020. 6
2020
-
[40]
Texture fields: Learning tex- ture representations in function space
Michael Oechsle, Lars Mescheder, Michael Niemeyer, Thilo Strauss, and Andreas Geiger. Texture fields: Learning tex- ture representations in function space. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4531–4540, 2019. 5
2019
-
[41]
Dinov2: Learning robust visual features without supervision
Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 2, 4, 7, 8
2023 arXiv
-
[42]
Deepsdf: Learning con- tinuous signed distance functions for shape representation
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning con- tinuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 165–174, 2019. 2, 3
2019
-
[43]
Nerfies: Deformable neural radiance fields
Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5865–5874, 2021. 3
2021
-
[44]
Pointnet++: Deep hierarchical feature learning on point sets in a metric space
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017. 6
2017
-
[45]
Com- mon objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction
Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Com- mon objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In Proceedings of the IEEE/CVF international conference on computer vi...
2021
-
[46]
Vision-as-inverse-graphics: Obtaining a rich 3d explanation of a scene from a single image
Lukasz Romaszko, Christopher KI Williams, Pol Moreno, and Pushmeet Kohli. Vision-as-inverse-graphics: Obtaining a rich 3d explanation of a scene from a single image. InPro- ceedings of the IEEE International Conference on Computer Vision Workshops, pages 851–859, 2017. 2, 3, 4
2017
-
[47]
High-resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2
2022
-
[49]
Markov chain monte carlo for automated face image analysis
Sandro Sch ¨onborn, Bernhard Egger, Andreas Morel-Forster, and Thomas Vetter. Markov chain monte carlo for automated face image analysis. International Journal of Computer Vi- sion, 123:160–183, 2017. 2, 3, 4
2017
-
[50]
Deep marching tetrahedra: a hybrid repre- sentation for high-resolution 3d shape synthesis
Tianchang Shen, Jun Gao, Kangxue Yin, Ming-Yu Liu, and Sanja Fidler. Deep marching tetrahedra: a hybrid repre- sentation for high-resolution 3d shape synthesis. Advances in Neural Information Processing Systems , 34:6087–6101,
-
[51]
Shic: Shape-image correspondences with no key- point supervision
Aleksandar Shtedritski, Christian Rupprecht, and Andrea Vedaldi. Shic: Shape-image correspondences with no key- point supervision. arXiv preprint arXiv:2407.18907, 2024. 2
2024 arXiv
-
[53]
Ponymation: Learning artic- ulated 3d animal motions from unlabeled online videos
Keqiang Sun, Dor Litvak, Yunzhi Zhang, Hongsheng Li, Ji- ajun Wu, and Shangzhe Wu. Ponymation: Learning artic- ulated 3d animal motions from unlabeled online videos. In European Conference on Computer Vision, pages 100–119. Springer, 2025. 3
2025
-
[54]
Topology-preserving shape recon- struction and registration via neural diffeomorphic flow
Shanlin Sun, Kun Han, Deying Kong, Hao Tang, Xiangyi Yan, and Xiaohui Xie. Topology-preserving shape recon- struction and registration via neural diffeomorphic flow. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 20845–20855, 2022. 3
2022
-
[55]
Mofa: Model-based deep convolutional face au- toencoder for unsupervised monocular reconstruction
Ayush Tewari, Michael Zollhofer, Hyeongwoo Kim, Pablo Garrido, Florian Bernard, Patrick Perez, and Christian Theobalt. Mofa: Model-based deep convolutional face au- toencoder for unsupervised monocular reconstruction. In Proceedings of the IEEE international conference on com-...
2017
-
[56]
Fml: Face model learning from videos
Ayush Tewari, Florian Bernard, Pablo Garrido, Gaurav Bharaj, Mohamed Elgharib, Hans-Peter Seidel, Patrick P´erez, Michael Zollhofer, and Christian Theobalt. Fml: Face model learning from videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti...
2019
-
[57]
Learning complete 3d morphable face models from images and videos
Ayush Tewari, Hans-Peter Seidel, Mohamed Elgharib, Chris- tian Theobalt, et al. Learning complete 3d morphable face models from images and videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3361–3371, 2021. 3
2021
-
[58]
Disentangled3d: Learn- ing a 3d generative model with disentangled geometry and appearance from monocular images
Ayush Tewari, Xingang Pan, Ohad Fried, Maneesh Agrawala, Christian Theobalt, et al. Disentangled3d: Learn- ing a 3d generative model with disentangled geometry and appearance from monocular images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogn...
2022
-
[59]
Learning category-specific deformable 3d mod- els for object reconstruction
Shubham Tulsiani, Abhishek Kar, Joao Carreira, and Jiten- dra Malik. Learning category-specific deformable 3d mod- els for object reconstruction. IEEE transactions on pattern analysis and machine intelligence, 39(4):719–731, 2016. 3
2016
-
[60]
Im- plicit mesh reconstruction from unannotated image collec- tions
Shubham Tulsiani, Nilesh Kulkarni, and Abhinav Gupta. Im- plicit mesh reconstruction from unannotated image collec- tions. arXiv preprint arXiv:2007.08504, 2020. 3
2007 arXiv
-
[61]
Robust object detection under occlusion with context- aware compositionalnets
Angtian Wang, Yihong Sun, Adam Kortylewski, and Alan L Yuille. Robust object detection under occlusion with context- aware compositionalnets. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 12645–12654, 2020. 7
2020
-
[62]
Nemo: Neural mesh models of contrastive features for robust 3d pose estimation
Angtian Wang, Adam Kortylewski, and Alan Yuille. Nemo: Neural mesh models of contrastive features for robust 3d pose estimation. arXiv preprint arXiv:2101.12378, 2021. 6
2021 arXiv
-
[63]
V oge: a differentiable volume renderer using gaussian ellipsoids for analysis-by-synthesis
Angtian Wang, Peng Wang, Jian Sun, Adam Kortylewski, and Alan Yuille. V oge: a differentiable volume renderer using gaussian ellipsoids for analysis-by-synthesis. arXiv preprint arXiv:2205.15401, 2022. 6, 7, 2
2022 arXiv
-
[64]
Dtf-net: Category-level pose estimation and shape reconstruction via deformable template field
Haowen Wang, Zhipeng Fan, Zhen Zhao, Zhengping Che, Zhiyuan Xu, Dong Liu, Feifei Feng, Yakun Huang, Xiuquan Qiao, and Jian Tang. Dtf-net: Category-level pose estimation and shape reconstruction via deformable template field. In Proceedings of the 31st ACM International Confere...
2023
-
[65]
Pixel2mesh: Generating 3d mesh models from single rgb images
Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. Pixel2mesh: Generating 3d mesh models from single rgb images. In Proceedings of the Euro- pean conference on computer vision (ECCV) , pages 52–67,
-
[66]
Unsupervised learning of probably symmetric deformable 3d objects from images in the wild
Shangzhe Wu, Christian Rupprecht, and Andrea Vedaldi. Unsupervised learning of probably symmetric deformable 3d objects from images in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1–10, 2020. 3
2020
-
[67]
Unsupervised learning of probably symmetric deformable 3d objects from images in the wild
Shangzhe Wu, Christian Rupprecht, and Andrea Vedaldi. Unsupervised learning of probably symmetric deformable 3d objects from images in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1–10, 2020. 2
2020
-
[68]
Dove: Learning deformable 3d objects by watching videos
Shangzhe Wu, Tomas Jakab, Christian Rupprecht, and An- drea Vedaldi. Dove: Learning deformable 3d objects by watching videos. International Journal of Computer Vision, 131(10):2623–2634, 2023. 2, 3
2023
-
[69]
Magicpony: Learning ar- ticulated 3d animals in the wild
Shangzhe Wu, Ruining Li, Tomas Jakab, Christian Rup- precht, and Andrea Vedaldi. Magicpony: Learning ar- ticulated 3d animals in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8792–8802, 2023. 2, 3, 4
2023
-
[70]
Beyond pascal: A benchmark for 3d object detection in the wild
Yu Xiang, Roozbeh Mottaghi, and Silvio Savarese. Beyond pascal: A benchmark for 3d object detection in the wild. In IEEE winter conference on applications of computer vision , pages 75–82. IEEE, 2014. 6
2014
-
[71]
Objectnet3d: A large scale database for 3d object recognition
Yu Xiang, Wonhui Kim, Wei Chen, Jingwei Ji, Christopher Choy, Hao Su, Roozbeh Mottaghi, Leonidas Guibas, and Sil- vio Savarese. Objectnet3d: A large scale database for 3d object recognition. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, Oc...
2016
-
[72]
Lasr: Learning articulated shape re- construction from a monocular video
Gengshan Yang, Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Huiwen Chang, Deva Ramanan, William T Freeman, and Ce Liu. Lasr: Learning articulated shape re- construction from a monocular video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patter...
2021
-
[73]
Viser: Video- specific surface embeddings for articulated 3d shape recon- struction
Gengshan Yang, Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Ce Liu, and Deva Ramanan. Viser: Video- specific surface embeddings for articulated 3d shape recon- struction. Advances in Neural Information Processing Sys- tems, 34:19326–19338, 2021. 3
2021
-
[74]
Banmo: Building animatable 3d neural models from many casual videos
Gengshan Yang, Minh V o, Natalia Neverova, Deva Ra- manan, Andrea Vedaldi, and Hanbyul Joo. Banmo: Building animatable 3d neural models from many casual videos. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition , pages 2863–2873, 2022. 2, 3
2022
-
[75]
Lassie: 11 Learning articulated shapes from sparse image ensemble via 3d part discovery
Chun-Han Yao, Wei-Chih Hung, Yuanzhen Li, Michael Ru- binstein, Ming-Hsuan Yang, and Varun Jampani. Lassie: 11 Learning articulated shapes from sparse image ensemble via 3d part discovery. Advances in Neural Information Process- ing Systems, 35:15296–15308, 2022. 3
2022
-
[76]
Hi-lassie: High-fidelity articulated shape and skeleton discovery from sparse image ensemble
Chun-Han Yao, Wei-Chih Hung, Yuanzhen Li, Michael Ru- binstein, Ming-Hsuan Yang, and Varun Jampani. Hi-lassie: High-fidelity articulated shape and skeleton discovery from sparse image ensemble. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...
2023
-
[77]
Vision as bayesian infer- ence: analysis by synthesis? Trends in cognitive sciences , 10(7):301–308, 2006
Alan Yuille and Daniel Kersten. Vision as bayesian infer- ence: analysis by synthesis? Trends in cognitive sciences , 10(7):301–308, 2006. 1
2006
-
[78]
Telling left from right: Identifying geometry-aware semantic corre- spondence
Junyi Zhang, Charles Herrmann, Junhwa Hur, Eric Chen, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. Telling left from right: Identifying geometry-aware semantic corre- spondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 307...
2024
-
[79]
A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence
Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Pola- nia Cabrera, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence. Advances in Neural Information Processing Systems, 36, 2024. 2, 4
2024
-
[80]
Self-supervised geomet- ric correspondence for category-level 6d object pose estima- tion in the wild
Kaifeng Zhang, Yang Fu, Shubhankar Borse, Hong Cai, Fatih Porikli, and Xiaolong Wang. Self-supervised geomet- ric correspondence for category-level 6d object pose estima- tion in the wild. arXiv preprint arXiv:2210.07199, 2022. 3
-
[81]
Seeing a rose in five thousand ways
Yunzhi Zhang, Shangzhe Wu, Noah Snavely, and Jiajun Wu. Seeing a rose in five thousand ways. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 962–971, 2023. 2
2023
-
[82]
Deep implicit templates for 3d shape representation
Zerong Zheng, Tao Yu, Qionghai Dai, and Yebin Liu. Deep implicit templates for 3d shape representation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1429–1439, 2021. 3, 4, 5
2021
-
[83]
Starmap for category-agnostic keypoint and viewpoint esti- mation
Xingyi Zhou, Arjun Karpur, Linjie Luo, and Qixing Huang. Starmap for category-agnostic keypoint and viewpoint esti- mation. In Proceedings of the European Conference on Com- puter Vision (ECCV), pages 318–334, 2018. 7, 2
2018
-
[84]
3d menagerie: Modeling the 3d shape and pose of animals
Silvia Zuffi, Angjoo Kanazawa, David W Jacobs, and Michael J Black. 3d menagerie: Modeling the 3d shape and pose of animals. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6365–6373,
-
[86]
7 we list our model’s parameters and their values
Parameters In Tab. 7 we list our model’s parameters and their values. We setσ as the average nearest neighbor distance for all vertices V to the sampled vertices V′, as follows σ = s 1 |V| X v∈V min v′∈V′ ||v− v′||2. (14)
-
[87]
Comprehensive Results We report all missing categorical quantitative results in Tabs. 8 to 10. Additional qualitative results are shown for ObjectNet3D in Fig. 4 and for SPair-71k in Fig. 6. 1 Method bicycle bus car chair couch motorcycle tv A VG Sup. StarMap [83] 83.2 94.4 90...
-
[2017]
The following parameters are set for training and testing our model
2 12 Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Space Supplementary Material Name Value Optimizer Adam Learning Rate 1.00E-04 Batch Size 12 Batch Accumulation 2 Losses λapp 1.00E-01 λCD 1.00E-01 λm 1.00E+00 λmdt 1.00E+02 λsdf...
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.