REVIEW 3 major objections 4 minor 99 references
DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstruction
T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper claims that predicting two point maps from a single image—one placing each pixel in the camera's 3D frame and the other on a canonical rest-pose model—reduces monocular 3D shape and pose reconstruction to a pixel-labeling task…
desk verdict A genuinely new dual point-map representation with strong real-image correspondence results, but the headline pose claim is never directly measured and the 3D benchmark is self-created and amended post-publication. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the pair of point maps $(P, Q)$ called DualPMs. $P$ assigns each pixel $u$ the 3D location $p$ of the visible surface point in the camera reference frame; $Q$ assigns the same pixel the location $q$ of the corresponding point on the object in a canonical rest pose. The pose is the difference field $P-Q$, so no separate pose regression is needed. For amodal completeness, the authors use a layered representation $(P^*, Q^*)$: each layer stores the next ray-object intersection behind the previous one, with an opacity map per layer, analogous to depth peeling. Learning is supervised with a self-calibrated $L^2$ loss with per-pixel confidence, and the canonical map $Q$ is predicted from features of a pretrained DINOv2-plus-Stable-Diffusion extractor, while the posed map $P$ is predicted conditional on $Q$ only, which the ablation shows improves out-of-distribution generalization.
What would settle it
A concrete test: train a DualPM model on a single synthetic horse model and evaluate on real images of a quadruped with a body plan far outside the horse template, such as a giraffe or a camel. If the canonical point map $Q$ fails to place anatomically matching points (for example, the knee joint) at consistent canonical coordinates across images, or if the predicted posed map $P$ no longer tracks the visible silhouette after the deformation $P-Q$ is applied, then the claimed category-level canonical space does not transfer. The paper's own appendix already documents failure on extreme viewpoints and shapes far from the training distribution, so the observable prediction is that these failures become systematic when the test category's topology differs from the training rig.
Extended reading notes
Core claim
The discovery is the DualPM representation itself: instead of predicting depth or a single 3D point map, the network predicts two point maps from the same image, $P$ in camera space and $Q$ in a canonical object space where the object is in a neutral rest pose. The deformation field $P-Q$ is then, by construction, the object's pose, and matching pixels across images reduces to matching canonical coordinates. A second contribution is the amodal layered point map, which associates each pixel with all ray-surface intersections, visible and occluded, so a full reconstruction is obtained despite self-occlusions. With this target, a convolutional network trained on roughly 30k synthetic renderings of one or two rigged 3D models per category outperforms prior methods on PASCAL VOC keypoint transfer (PCK@0.1) and on the Animodel-Points Chamfer distance benchmarks, and generalizes to real images of horses, cows, sheep, and even unseen categories.
Load-bearing premise
The method assumes that one canonical rest-pose coordinate space can be shared by an entire object category, an anchor fixed by the one or two rigged 3D models used in training, and that the pretrained image features map any real instance into that same canonical space; the paper itself notes that aligning canonical spaces of multiple 3D models remains an unsolved problem, so if the canonical space is inconsistent across instances, $Q$ is ill-defined and $P-Q$ no longer encodes pose.
Editorial extensions
If this is right
- Any geometric task for deformable objects—keypoint transfer, dense correspondence, scene flow between poses, skeleton fitting, and motion retargeting—reduces to reading off the two point maps, since pose is the difference $P-Q$ and correspondence is equality of canonical coordinates.
- Training on one or two synthetic 3D models per category suffices to generalize to real images, outperforming self-supervised methods trained on large real-image collections such as 3D-Fauna and large-3D-asset methods such as Trellis on the reported benchmarks.
- The amodal layered representation recovers the complete 3D shape of the object through self-occlusions, a capability that single-layer point maps lack.
- The canonical point map $Q$ doubles as a useful image feature: conditioning $P$ on $Q$ rather than on raw image features improves out-of-distribution generalization, per the ablation study.
Reading between the lines
- Because the pose is read out as a per-pixel difference field, the same representation should transfer to other articulated categories such as humans, hands, or birds without architectural changes, provided a rigged 3D model defines the category's canonical space.
- The unsolved canonical-space alignment across multiple 3D models (acknowledged in App. B) suggests that the method's scalability depends on learning a shared canonical space; a testable direction is to fit DualPMs jointly to multiple rigs via a learned warp, which would let training data grow beyond one or two models.
- The paper predicts only the expectation over unseen, self-occluded regions; a natural extension is to predict a distribution or multiple hypotheses per layer, which would address the unrealistic reconstructions the authors document for invisible parts.
- The zero-shot generalization from a horse-trained model to cows and sheep hints that quadrupeds share a sufficiently aligned canonical structure; this suggests a testable scaling law: adding a few more rigged species should extend the model to any quadruped, and possibly to any articulated object, without retraining.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces DualPM, a representation for monocular 3D reconstruction of deformable objects: for each pixel, the network predicts a posed point map P in camera space and a canonical point map Q in a rest-pose object frame, with the pose encoded as the flow P − Q. It also extends point maps to amodal reconstruction via layered outputs that capture occluded surfaces. Training is done purely on synthetic renderings from one or two rigged 3D models per category from Animodel, using DINOv2 and Stable Diffusion features for the canonical predictor and conditioning the posed predictor on Q. Experiments report PCK on PASCAL VOC, Chamfer distance on the newly introduced Animodel-Points benchmark, ablations on conditioning and layer count, and qualitative skeleton fitting and animation results.
Significance. The representation is clearly formulated and the reported quantitative gains are substantial: PCK@0.1 on PASCAL VOC horses improves from 53.9 for 3D-Fauna to 73.2, and Chamfer distances on the introduced benchmark are much lower than existing methods. If the pose-decoding claim were validated, DualPM would be a simple and network-friendly target for deformable object reconstruction, and the synthetic-to-real generalization story would be notable. The paper also gives useful ablations showing the benefit of conditioning P on Q rather than on image features. However, the central pose claim is not yet empirically demonstrated, and the main 3D evaluation is partly based on a benchmark constructed by the authors from the same data family used for training. The independent PASCAL VOC result supports correspondence quality, but not the quantitative pose interpretation of P − Q.
major comments (3)
- [Sec. 3.1 and Fig. 1; Sec. 4.2–4.3] The paper defines the deformation field P − Q as the pose, but no experiment measures whether P − Q agrees with ground-truth articulation. Table 1 reports PCK, which evaluates the canonical map Q through 2D correspondences, and Chamfer distance of the posed point map P, which measures shape. Sec. 4.3 presents skeleton fitting only qualitatively (Figs. 1 and 6) and reports no joint-angle or bone-transform errors. Because the central claim is that 3D pose estimation reduces to DualPM prediction, the absence of any pose-error metric is a load-bearing gap. I would expect at least an evaluation on held-out synthetic poses from a rigged model where ground-truth joint rotations are known, reporting per-bone rotation error or a comparable pose metric, separately for seen and unseen templates.
- [App. B; Sec. 3.1] The interpretation of P − Q as pose relies on Q being defined in a single canonical space shared by all instances. App. B states that aligning canonical spaces across additional 3D models is unsolved, and training uses one or two models per category. For a novel instance such as a foal or donkey (Fig. 5), Q is anchored to the training template's rest pose, so P − Q contains body-shape and identity differences in addition to articulation. The paper does not isolate pose from shape. A quantitative experiment comparing predicted P − Q against an instance-specific canonical rest pose, or another explicit decomposition of shape and pose, is needed to support the pose claim; until then, the phrase 'shape and pose reconstruction' overstates what is measured.
- [App. D; Tab. 1] The main 3D quantitative benchmark, Animodel-Points, is introduced by this paper and is derived from the same Animodel data family used for training. This limits the strength of the cross-method shape-reconstruction comparison: the test distribution is close to the training distribution, and several baselines are the authors' own methods. The PASCAL VOC PCK result is independent and supports the correspondence/canonical-map claim, but it does not measure 3D shape accuracy. I recommend either re-running the shape comparison on an external 3D dataset or clearly separating the self-created benchmark results from the claim of general 3D reconstruction superiority; as it stands, the headline 'outperforms prior methods for 3D reconstruction' rests partly on a benchmark constructed from the training distribution.
minor comments (4)
- [Sec. 4.4] The first sentence says the ablation results are reported 'in Sec. 4.4,' which is the current section; it should point to Table 2.
- [Sec. 4.2] The inline reference 'Tab. 11' appears to be a typo for Table 1, and the footnote marker is rendered confusingly in the sentence 'The performance of our method and others is reported in Tab. 11'.
- [Sec. 3.2] The opacity loss Lσ is mentioned but its exact form is not given; since the layered amodal supervision is a contribution, please specify the loss explicitly.
- [References] The same work 'MagicPony' appears to be cited as both [70] and [72] in the text; please make the citation consistent or clarify that these are distinct versions.
Circularity Check
No significant circularity: the DualPM pose reduction is an explicit representation definition rather than a hidden reuse of fitted inputs, and the main generalization claims are grounded in external PASCAL VOC evaluations.
full rationale
The paper's central construction is a representation reduction: pose is defined as the deformation field P−Q (Sec. 1: 'the deformation field that we wish to recover is simply the difference between the two point maps'), so the claim that pose estimation reduces to DualPM prediction follows from the definition. This is a design choice rather than a circular derivation, because P and Q are image-conditioned network outputs trained against rendered ground-truth point maps, not quantities refit to the evaluation target. The key empirical claim, that networks trained on one or two synthetic models generalize to real images, is supported by the external PASCAL VOC PCK@0.1 benchmark, which is independent of the authors' training data and evaluation pipeline. The Animodel-Points benchmark (App. D) is derived from the authors' Farm3D data and several baselines (Farm3D, 3D-Fauna, MagicPony) are from the same group, but this is ordinary use of published resources rather than load-bearing circularity; the benchmark explicitly excludes training poses. App. B's admission that canonical-space alignment across multiple 3D models is unsolved is a validity limitation for the pose interpretation on novel identities, not a circular step, since no fitted parameter is renamed as a prediction. The absence of a direct quantitative pose-accuracy metric is a correctness and validation gap, not a circularity of the derivation chain.
Assumptions & free parameters
free parameters (5)
- number of predicted layers K =
4 (4 ray pairs, 8 layers)
- confidence loss log-weight alpha =
not reported
- feature dimension after PCA =
64
- training schedule =
100k steps, batch size 12, LR 6e-4, step decay 30k with factor 0.5
- point map resolution =
160x160
assumptions (4)
- domain assumption Pre-trained DINOv2 and Stable Diffusion features, combined as in [86], provide pose- and deformation-invariant features that serve as a proxy for the canonical point map.
- domain assumption A single canonical space can be defined for an entire object category, anchored by the one or two rigged models used for training.
- domain assumption The camera is always outside the object, so every viewing ray intersects the surface an even number of times.
- domain assumption An accurate object segmentation mask M is available at inference time.
invented entities (2)
-
Dual Point Maps (P, Q)
independent evidence
-
Layered amodal point maps (P*, Q*)
independent evidence
Cite this review
Pith. "Pith review of DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstruction." pith.science (2026). https://pith.science/paper/UI6DSOEL
@misc{pith2026241204464,
author = {Pith},
title = {Pith review of: DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstruction},
year = {2026},
howpublished = {\url{https://pith.science/paper/UI6DSOEL}},
note = {Machine review of arXiv:2412.04464}
}
read the original abstract
The choice of data representation is a key factor in the success of deep learning in geometric tasks. For instance, DUSt3R recently introduced the concept of viewpoint-invariant point maps, generalizing depth prediction and showing that all key problems in the 3D reconstruction of static scenes can be reduced to predicting such point maps. In this paper, we develop an analogous concept for a very different problem: the reconstruction of the 3D shape and pose of deformable objects. To this end, we introduce Dual Point Maps (DualPM), where a pair of point maps is extracted from the same image-one associating pixels to their 3D locations on the object and the other to a canonical version of the object in its rest pose. We also extend point maps to amodal reconstruction to recover the complete shape of the object, even through self-occlusions. We show that 3D reconstruction and 3D pose estimation can be reduced to the prediction of DualPMs. Empirically, we demonstrate that this representation is a suitable target for deep networks to predict. Specifically, we focus on modeling quadrupeds, showing that DualPMs can be trained purely on synthetic 3D data, consisting of one or two models per category, while generalizing effectively to real images. With this approach, we achieve significant improvements over previous methods for the 3D analysis and reconstruction of such objects.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
AutoAvatar: Au- toregressive neural fields for dynamic avatar modeling
Ziqian Bai, Timur Bagautdinov, Javier Romero, Michael Zollh¨ofer, Ping Tan, and Shunsuke Saito. AutoAvatar: Au- toregressive neural fields for dynamic avatar modeling. In Proc. ECCV, 2022
2022
-
[2]
Narasimhan
Aayush Bansal, Minh V o, Yaser Sheikh, Deva Ramanan, and Srinivasa G. Narasimhan. 4D visualization of dynamic events from unconstrained multi-view videos. In Proc. CVPR, 2020
2020
-
[3]
Fitzgibbon, and Roberto Cipolla
Benjamin Biggs, Thomas Roddick, Andrew W. Fitzgibbon, and Roberto Cipolla. Creatures great and SMAL: recovering the shape and motion of animals from video. In Proc. ECCV, 2018
2018
-
[4]
Goat image dataset
Masum Billah, Yu Jiantao, and Yu Jiang. Goat image dataset. Mendeley Data, V2, 2020
2020
-
[5]
Hexplane: A fast representation for dynamic scenes
Ang Cao and Justin Johnson. Hexplane: A fast representation for dynamic scenes. arXiv.cs, abs/2301.09632, 2023
arXiv 2023
-
[6]
Relighting4D: Neural relightable human from videos
Zhaoxi Chen and Ziwei Liu. Relighting4D: Neural relightable human from videos. In Proc. ECCV, 2022
2022
-
[7]
Dream- Scene4D: dynamic multi-object scene generation from monocular videos
Wen-Hsuan Chu, Lei Ke, and Katerina Fragkiadaki. Dream- Scene4D: dynamic multi-object scene generation from monocular videos. arXiv, 2405.02280, 2024
arXiv 2024
-
[8]
Tenen- baum, and Jiajun Wu
Yilun Du, Yinan Zhang, Hong-Xing Yu, Joshua B. Tenen- baum, and Jiajun Wu. Neural radiance flow for 4d view synthesis and video processing. In Proc. ICCV, 2021
2021
Show all 99 references
-
[9]
MASt3R- SfM: a fully-integrated solution for unconstrained structure- from-motion
Bardienus Duisterhof, Lojze Zust, Philippe Weinzaepfel, Vin- cent Leroy, Yohann Cabon, and Jerome Revaud. MASt3R- SfM: a fully-integrated solution for unconstrained structure- from-motion. arXiv, 2409.19152, 2024
2024 arXiv
-
[10]
Duisterhof, Zhao Mandi, Yunchao Yao, Jia-Wei Liu, Mike Zheng Shou, Shuran Song, and Jeffrey Ichnowski
Bardienus P. Duisterhof, Zhao Mandi, Yunchao Yao, Jia-Wei Liu, Mike Zheng Shou, Shuran Song, and Jeffrey Ichnowski. MD-Splatting: Learning metric deformation from 4d gaus- sians in highly deformable scenes. arXiv.cs, 2312.00583, 2023
2023 arXiv
-
[11]
Taming transformers for high-resolution image synthesis
Patrick Esser, Robin Rombach, and Bj¨orn Ommer. Taming transformers for high-resolution image synthesis. In Proc. CVPR, 2021
2021
-
[12]
Everingham, L
M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) challenge. IJCV, 88(2):303–338, 2010
2010
-
[13]
Gaussian splashing: Dynamic fluid synthesis with gaussian splatting
Yutao Feng, Xiang Feng, Yintong Shang, Ying Jiang, Chang Yu, Zeshun Zong, Tianjia Shao, Hongzhi Wu, Kun Zhou, Chenfanfu Jiang, and Yin Yang. Gaussian splashing: Dynamic fluid synthesis with gaussian splatting. arXiv.cs, 2401.15318, 2024
2024 arXiv
-
[14]
K-planes: Explicit radiance fields in space, time, and appearance
Sara Fridovich-Keil, Giacomo Meanti, Frederik Warburg, Benjamin Recht, and Angjoo Kanazawa. K-planes: Explicit radiance fields in space, time, and appearance. arXiv.cs, abs/2301.10241, 2023
2023 arXiv
-
[15]
Dynamic neural radiance fields for monocular 4d facial avatar reconstruction
Guy Gafni, Justus Thies, Michael Zollh ¨ofer, and Matthias Nießner. Dynamic neural radiance fields for monocular 4d facial avatar reconstruction. In Proc. CVPR, 2021
2021
-
[16]
Dynamic view synthesis from dynamic monocular video
Chen Gao, Ayush Saraf, Johannes Kopf, and Jia-Bin Huang. Dynamic view synthesis from dynamic monocular video. In Proc. ICCV, 2021
2021
-
[17]
Shape and viewpoints without keypoints
Shubham Goel, Angjoo Kanazawa, and Jitendra Malik. Shape and viewpoints without keypoints. In Proc. ECCV, 2020
2020
-
[18]
Farm3D: Learning articulated 3D ani- mals by distilling 2D diffusion
Tomas Jakab, Ruining Li, Shangzhe Wu, Christian Rupprecht, and Andrea Vedaldi. Farm3D: Learning articulated 3D ani- mals by distilling 2D diffusion. In Proc. 3DV, 2024
2024
-
[19]
VR-GS: A physical dynamics-aware interactive gaussian splatting system in virtual reality
Ying Jiang, Chang Yu, Tianyi Xie, Xuan Li, Yutao Feng, Huamin Wang, Minchen Li, Henry Lau, Feng Gao, Yin Yang, and Chenfanfu Jiang. VR-GS: A physical dynamics-aware interactive gaussian splatting system in virtual reality. arXiv, 2401.16663, 2024
2024 arXiv
-
[20]
What uncertainties do we need in Bayesian deep learning for computer vision? Proc
Alex Kendall and Yarin Gal. What uncertainties do we need in Bayesian deep learning for computer vision? Proc. NeurIPS, 2017
2017
-
[21]
3D Gaussian Splatting for real-time radi- ance field rendering
Bernhard Kerbl, Georgios Kopanas, Thomas Leimk¨uhler, and George Drettakis. 3D Gaussian Splatting for real-time radi- ance field rendering. Proc. SIGGRAPH, 42(4), 2023
2023
-
[22]
Kingma and Jimmy Ba
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Proc. ICLR, 2015
2015
-
[23]
Berg, Wan-Yen Lo, Piotr Doll´ar, and Ross Girshick
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll´ar, and Ross Girshick. Segment anything. In Proc. CVPR, 2023
2023
-
[24]
DynMF: Neural motion factorization for real-time dynamic view synthesis with 3D Gaussian splatting
Agelos Kratimenos, Jiahui Lei, and Kostas Daniilidis. DynMF: Neural motion factorization for real-time dynamic view synthesis with 3D Gaussian splatting. arXiv.cs, abs/2312.00112, 2023
2023 arXiv
-
[25]
Canonical surface mapping via geometric cycle consistency
Nilesh Kulkarni, Abhinav Gupta, and Shubham Tulsiani. Canonical surface mapping via geometric cycle consistency. In Proc. ICCV, 2019
2019
-
[26]
Fouhey, and Shub- ham Tulsiani
Nilesh Kulkarni, Abhinav Gupta, David F. Fouhey, and Shub- ham Tulsiani. Articulation-aware canonical surface mapping. In Proc. CVPR, pages 449–458, 2020
2020
-
[27]
Monocular dense 3D reconstruction of a complex dynamic scene from two perspective frames
Suryansh Kumar, Yuchao Dai, and Hongdong Li. Monocular dense 3D reconstruction of a complex dynamic scene from two perspective frames. In Proc. ICCV, 2017
2017
-
[28]
MoSca: dynamic gaussian fusion from casual videos via 4d motion scaffolds
Jiahui Lei, Yijia Weng, Adam Harley, Leonidas Guibas, and Kostas Daniilidis. MoSca: dynamic gaussian fusion from casual videos via 4d motion scaffolds. arXiv, 2405.17421, 2024
2024 arXiv
-
[29]
Dessie: Disentanglement for articulated 3D horse shape and pose estimation from images
Ci Li, Yi Yang, Zehang Weng, Elin Hernlund, Silvia Zuffi, and Hedvig Kjellstr¨om. Dessie: Disentanglement for articulated 3D horse shape and pose estimation from images. arXiv, 2410.03438, 2024
2024 arXiv
-
[30]
Self-supervised single-view 3D reconstruction via semantic consistency
Xueting Li, Sifei Liu, Kihwan Kim, Shalini De Mello, Varun Jampani, Ming-Hsuan Yang, and Jan Kautz. Self-supervised single-view 3D reconstruction via semantic consistency. In Proc. ECCV, 2020
2020
-
[31]
Neural scene flow fields for space-time view synthesis of dynamic scenes
Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space-time view synthesis of dynamic scenes. In Proc. CVPR, 2021
2021
-
[32]
DynIBaR: Neural dynamic image-based rendering
Zhengqi Li, Qianqian Wang, Forrester Cole, Richard Tucker, and Noah Snavely. DynIBaR: Neural dynamic image-based rendering. In Proc. CVPR, 2023. 9
2023
-
[33]
Learning the 3D fauna of the Web
Zizhang Li, Dor Litvak, Ruining Li, Yunzhi Zhang, Tomas Jakab, Christian Rupprecht, Shangzhe Wu, Andrea Vedaldi, and Jiajun Wu. Learning the 3D fauna of the Web. In Proc. CVPR, 2024
2024
-
[34]
GauFRe: Gaussian deformation fields for real-time dynamic novel view synthesis
Yiqing Liang, Numair Khan, Zhengqin Li, Thu Nguyen- Phuoc, Douglas Lanman, James Tompkin, and Lei Xiao. GauFRe: Gaussian deformation fields for real-time dynamic novel view synthesis. arXiv.cs, abs/2312.11458, 2023
2023 arXiv
-
[35]
Im4d: High-fidelity and real- time novel view synthesis for dynamic scenes
Haotong Lin, Sida Peng, Zhen Xu, Tao Xie, Xingyi He, Hu- jun Bao, and Xiaowei Zhou. Im4d: High-fidelity and real- time novel view synthesis for dynamic scenes. In Proc. SIG- GRAPH Asia, 2023
2023
-
[36]
Gaussian- flow: 4d reconstruction with dynamic 3d gaussian particle
Youtian Lin, Zuozhuo Dai, Siyu Zhu, and Yao Yao. Gaussian- flow: 4d reconstruction with dynamic 3d gaussian particle. In Proc. CVPR, 2024
2024
-
[37]
Align your gaussians: Text-to-4d with dynamic 3d gaussians and composed diffusion models
Huan Ling, Seung Wook Kim, Antonio Torralba, Sanja Fidler, and Karsten Kreis. Align your gaussians: Text-to-4d with dynamic 3d gaussians and composed diffusion models. In Proc. CVPR, 2024
2024
-
[38]
Dynamic gaussians mesh: Consistent mesh reconstruction from monocular videos
Isabella Liu, Hao Su, and Xiaolong Wang. Dynamic gaussians mesh: Consistent mesh reconstruction from monocular videos. arXiv, 2404.12379, 2024
2024 arXiv
-
[39]
Neural human video rendering: Joint learning of dynamic textures and rendering-to-video transla- tion
Lingjie Liu, Weipeng Xu, Marc Habermann, Michael Zollh¨ofer, Florian Bernard, Hyeongwoo Kim, Wenping Wang, and Christian Theobalt. Neural human video rendering: Joint learning of dynamic textures and rendering-to-video transla- tion. arXiv.cs, abs/2001.04947, 2020
2001 arXiv
-
[40]
Saragih, Gabriel Schwartz, Andreas M
Stephen Lombardi, Tomas Simon, Jason M. Saragih, Gabriel Schwartz, Andreas M. Lehrmann, and Yaser Sheikh. Neural volumes: learning dynamic renderable volumes from images. ACM Trans. Graph., 38(4), 2019
2019
-
[41]
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. SMPL: a skinned multi- person linear model. ACM Trans. on Graphics (TOG), 2015
2015
-
[42]
Srinivasan, Matthew Tancik, Jonathan T
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthe- sis. In Proc. ECCV, 2020
2020
-
[43]
Newcombe, Dieter Fox, and Steven M
Richard A. Newcombe, Dieter Fox, and Steven M. Seitz. DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time. In Proc. CVPR, 2015
2015
-
[44]
Capturing the geometry of object categories from video supervision
David Novotn´y, Diane Larlus, and Andrea Vedaldi. Capturing the geometry of object categories from video supervision. 2018
2018
-
[45]
Maxime Oquab, Timoth´ee Darcet, Th´eo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido As- sran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po- Yao Huang, Shang-Wen Li, Ishan Misra, Michae...
2024
-
[46]
Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans
Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. In Proc. CVPR, 2021
2021
-
[47]
D-NeRF: Neural radiance fields for dynamic scenes
Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-NeRF: Neural radiance fields for dynamic scenes. In Proc. CVPR, 2021
2021
-
[48]
L4GM: Large 4D Gaussian reconstruction model
Jiawei Ren, Kevin Xie, Ashkan Mirzaei, Hanxue Liang, Xiao- hui Zeng, Karsten Kreis, Ziwei Liu, Antonio Torralba, Sanja Fidler, Seung Wook Kim, and Huan Ling. L4GM: Large 4D Gaussian reconstruction model. arXiv, 2406.10324, 2024
2024 arXiv
-
[49]
High-resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proc. CVPR, 2022
2022
-
[50]
Barc: Learning to regress 3d dog shape from images by exploiting breed information
Nadine R¨uegg, Silvia Zuffi, Konrad Schindler, and Michael J Black. Barc: Learning to regress 3d dog shape from images by exploiting breed information. In Proc. CVPR, 2022
2022
-
[51]
Video pop-up: Monocular 3D reconstruction of dynamic scenes
Chris Russell, Rui Yu, and Lourdes Agapito. Video pop-up: Monocular 3D reconstruction of dynamic scenes. In Proc. ECCV, 2014
2014
-
[52]
Freeman, Fr´edo Durand, Joshua B
Prafull Sharma, Ayush Tewari, Yilun Du, Sergey Zakharov, Rares Ambrus, Adrien Gaidon, William T. Freeman, Fr´edo Durand, Joshua B. Tenenbaum, and Vincent Sitzmann. Seeing 3d objects in a single image via self-supervised static-dynamic disentanglement. arXiv.cs, abs/2207.11232, 2022
2022 arXiv
-
[53]
Learning dynamic facial radiance fields for few-shot talking head synthesis
Shuai Shen, Wanhua Li, Zheng Zhu, Yueqi Duan, Jie Zhou, and Jiwen Lu. Learning dynamic facial radiance fields for few-shot talking head synthesis. In Proc. ECCV, 2022
2022
-
[54]
NeRF- Player: A streamable dynamic scene representation with de- composed neural radiance fields
Liangchen Song, Anpei Chen, Zhong Li, Zhang Chen, Lele Chen, Junsong Yuan, Yi Xu, and Andreas Geiger. NeRF- Player: A streamable dynamic scene representation with de- composed neural radiance fields. IEEE Trans. on Visualiza- tion and Computer Graphics, 29(5), 2023
2023
-
[55]
Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Proc. ICLR, 2021
2021
-
[56]
Dynamic Gaussian marbles for novel view synthesis of casual monocular videos
Colton Stearns, Adam Harley, Mikaela Uy, Florian Dubost, Federico Tombari, Gordon Wetzstein, and Leonidas Guibas. Dynamic Gaussian marbles for novel view synthesis of casual monocular videos. arXiv, 2406.18717, 2024
2024 arXiv
-
[57]
Unsuper- vised learning of object frames by dense equivariant image labelling
James Thewlis, Hakan Bilen, and Andrea Vedaldi. Unsuper- vised learning of object frames by dense equivariant image labelling. In Proc. NeurIPS, 2017
2017
-
[58]
Unsu- pervised learning of object landmarks by factorized spatial embeddings
James Thewlis, Hakan Bilen, and Andrea Vedaldi. Unsu- pervised learning of object landmarks by factorized spatial embeddings. In Proc. ICCV, 2017
2017
-
[59]
Modelling and unsupervised learning of symmetric deformable object categories
James Thewlis, Hakan Bilen, and Andrea Vedaldi. Modelling and unsupervised learning of symmetric deformable object categories. In Proc. NeurIPS, 2018
2018
-
[60]
He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, and Leonidas J. Guibas. Normalized object coordinate space for category-level 6d object pose and size estimation. In Proc. CVPR, pages 2642–2651, 2019
2019
-
[61]
Mor- pheus: Neural dynamic 360 ° surface reconstruction from monocular RGB-D video
Hengyi Wang, Jingwen Wang, and Lourdes Agapito. Mor- pheus: Neural dynamic 360 ° surface reconstruction from monocular RGB-D video. arXiv.cs, abs/2312.00778, 2023
2023 arXiv
-
[62]
PoseDiffusion: solving pose estimation via diffusion-aided bundle adjustment
Jianyuan Wang, Christian Rupprecht, and David Novotny. PoseDiffusion: solving pose estimation via diffusion-aided bundle adjustment. In Proc. ICCV, 2023. 10
2023
-
[63]
VGGSfM: visual geometry grounded deep structure from motion
Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny. VGGSfM: visual geometry grounded deep structure from motion. In Proc. CVPR, 2024
2024
-
[64]
Fourier PlenOctrees for dynamic radiance field rendering in real-time
Liao Wang, Jiakai Zhang, Xinhang Liu, Fuqiang Zhao, Yan- shun Zhang, Yingliang Zhang, Minye Wu, Jingyi Yu, and Lan Xu. Fourier PlenOctrees for dynamic radiance field rendering in real-time. In Proc. CVPR, 2022
2022
-
[65]
Shape of motion: 4D reconstruc- tion from a single video
Qianqian Wang, Vickie Ye, Hang Gao, Jake Austin, Zhengqi Li, and Angjoo Kanazawa. Shape of motion: 4D reconstruc- tion from a single video. arXiv, 2407.13764, 2024
2024
-
[66]
DUSt3R: Geometric 3D vision made easy
Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. DUSt3R: Geometric 3D vision made easy. In Proc. CVPR, 2024
2024
-
[67]
MonoRec: Semi-supervised dense reconstruction in dynamic environments from a sin- gle moving camera
Felix Wimbauer, Nan Yang, Lukas von Stumberg, Niclas Zeller, and Daniel Cremers. MonoRec: Semi-supervised dense reconstruction in dynamic environments from a sin- gle moving camera. In Proc. CVPR, 2021
2021
-
[68]
4D gaussian splatting for real-time dynamic scene rendering
Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4D gaussian splatting for real-time dynamic scene rendering. In Proc. CVPR, 2023
2023
-
[69]
Un- supervised learning of probably symmetric deformable 3D objects from images in the wild
Shangzhe Wu, Christian Rupprecht, and Andrea Vedaldi. Un- supervised learning of probably symmetric deformable 3D objects from images in the wild. In Proc. CVPR, 2020
2020
-
[70]
Magicpony: Learning articulated 3d animals in the wild
Shangzhe Wu, Ruining Li, Tomas Jakab, Christian Rupprecht, and Andrea Vedaldi. Magicpony: Learning articulated 3d animals in the wild. arXiv.cs, 2022
2022
-
[71]
DOVE: Learning deformable 3D objects by watching videos
Shangzhe Wu, Tomas Jakab, Christian Rupprecht, and Andrea Vedaldi. DOVE: Learning deformable 3D objects by watching videos. In IJCV, 2023
2023
-
[72]
MagicPony: Learning articulated 3D animals in the wild
Shangzhe Wu, Ruining Li, Tomas Jakab, Christian Rupprecht, and Andrea Vedaldi. MagicPony: Learning articulated 3D animals in the wild. In Proc. CVPR, 2023
2023
-
[73]
Structured 3D latents for scalable and versatile 3D generation
Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3D latents for scalable and versatile 3D generation. arXiv, 2412.01506, 2024
2024 arXiv
-
[74]
PhysGaussian: Physics- integrated 3D Gaussians for generative dynamics
Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. PhysGaussian: Physics- integrated 3D Gaussians for generative dynamics. arXiv.cs, abs/2311.12198, 2023
2023 arXiv
-
[75]
Street gaus- sians for modeling dynamic urban scenes
Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, and Xianpeng Lang. Street gaus- sians for modeling dynamic urban scenes. arXiv, 2024
2024
-
[76]
Freeman, and Ce Liu
Gengshan Yang, Deqing Sun2, Varun Jampani, Daniel Vlasic, Forrester Cole, Huiwen Chang, Deva Ramanan, William T. Freeman, and Ce Liu. LASR: Learning articulated shape reconstruction from a monocular video. In Proc. CVPR, 2021
2021
-
[77]
BANMo: Building ani- matable 3d neural models from many casual videos
Gengshan Yang, Minh V o, Natalia Neverova, Deva Ramanan, Andrea Vedaldi, and Hanbyul Joo. BANMo: Building ani- matable 3d neural models from many casual videos. In Proc. CVPR, 2022
2022
-
[78]
Dinesh Reddy, and Deva Ramanan
Gengshan Yang, Chaoyang Wang, N. Dinesh Reddy, and Deva Ramanan. Reconstructing animatable categories from videos. In Proc. CVPR, 2023
2023
-
[79]
Depth anything V2
Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything V2. arXiv, 2406.09414, 2024
2024 arXiv
-
[80]
Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting
Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting. In Proc. ICLR, 2024
2024
-
[81]
LASSIE: learning articulated shapes from sparse image ensemble via 3d part discovery
Chun-Han Yao, Wei-Chih Hung, Yuanzhen Li, Michael Ru- binstein, Ming-Hsuan Yang, and Varun Jampani. LASSIE: learning articulated shapes from sparse image ensemble via 3d part discovery. arXiv.cs, abs/2207.03434, 2022
2022 arXiv
-
[82]
ARTIC3D: Learning robust articulated 3D shapes from noisy web image collections
Chun-Han Yao, Amit Raj, Wei-Chih Hung, Yuanzhen Li, Michael Rubinstein, Ming-Hsuan Yang, and Varun Jampani. ARTIC3D: Learning robust articulated 3D shapes from noisy web image collections. Proc. NeurIPS, 2023
2023
-
[83]
4DGen: Grounded 4D content generation with spatial-temporal consistency
Yuyang Yin1, Dejia Xu2, Zhangyang Wang, Yao Zhao, and Yunchao Wei. 4DGen: Grounded 4D content generation with spatial-temporal consistency. arXiv.cs, 2023
2023
-
[84]
Novel view synthesis of dynamic scenes with globally coherent depths from a monocular camera
Jae Shin Yoon, Kihwan Kim, Orazio Gallo, Hyun Soo Park, and Jan Kautz. Novel view synthesis of dynamic scenes with globally coherent depths from a monocular camera. In Proc. CVPR, 2020
2020
-
[85]
EgoGaussian: Dynamic scene understanding from egocentric video with 3D gaussian splatting
Daiwei Zhang, Gengyan Li, Jiajie Li, Micka ¨el Bressieux, Otmar Hilliges, Marc Pollefeys, Luc Van Gool, and Xi Wang. EgoGaussian: Dynamic scene understanding from egocentric video with 3D gaussian splatting. arXiv.cs, abs/2406.19811, 2024
2024 arXiv
-
[86]
A tale of two features: Stable diffusion complements DINO for zero-shot semantic correspondence
Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Pola- nia Cabrera, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. A tale of two features: Stable diffusion complements DINO for zero-shot semantic correspondence. arXiv.cs, abs/2305.15347, 2023
2023 arXiv
-
[87]
MonST3R: a simple approach for estimating geometry in the presence of motion
Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. MonST3R: a simple approach for estimating geometry in the presence of motion. arXiv, 2410.03825, 2024
2024 arXiv
-
[88]
Animate124: Animating one image to 4D dynamic scene
Yuyang Zhao, Zhiwen Yan, Enze Xie, Lanqing Hong, Zhen- guo Li, and Gim Hee Lee. Animate124: Animating one image to 4D dynamic scene. arXiv.cs, abs/2311.14603, 2023
2023 arXiv
-
[89]
Jacobs, and Michael J
Silvia Zuffi, Angjoo Kanazawa, David W. Jacobs, and Michael J. Black. 3D menagerie: Modeling the 3D shape and pose of animals. In Proc. CVPR, 2017
2017
-
[90]
Lions and tigers and bears: Capturing non-rigid, 3d, articulated shape from images
Silvia Zuffi, Angjoo Kanazawa, and Michael J Black. Lions and tigers and bears: Capturing non-rigid, 3d, articulated shape from images. In Proc. CVPR, 2018
2018
-
[91]
Varen: Very accurate and realistic equine network
Silvia Zuffi, Ylva Mellbin, Ci Li, Markus Hoeschle, Hedvig Kjellstr¨om, Senya Polikovsky, Elin Hernlund, and Michael J Black. Varen: Very accurate and realistic equine network. In Proc. CVPR, pages 5374–5383, 2024. 11 Appendix A. Generalization to unseen categories Given the g...
2024
-
[92]
Coordinate transformation: The source mesh is trans- formed into model-view coordinates
-
[93]
Uniform scaling: The mesh is uniformly scaled by a fac- tor of V −1/3, where V is the mesh volume, to normalize its size
-
[94]
Evaluation protocol For a given generated point cloud, the following evaluation steps are performed:
Point sampling: 20,000 points are uniformly sampled from the surface of the scaled mesh to serve as the ground truth target. Evaluation protocol For a given generated point cloud, the following evaluation steps are performed:
-
[95]
Resampling: If the input point cloud does not contain 20,000 points, it is resampled to this size
-
[96]
Rotational ambiguity handling: The 20,000 point sam- ple is duplicated, and the duplicate is rotated 180 degrees around its vertical axis
-
[97]
Evaluation on Animodel
Subsampling for fitting: A subset of 10,000 points is sampled from the input to be used in the alignment pro- 13 Method Chamfer Distance (cm) Real-Sized Normalized Horse Cow Sheep Horse Cow Sheep A-CSM [26] 7.60 ± 3.07 6.71 ± 1.81 2.84 ± 0.77 2.73 ± 1.13 2.35 ± 0.68 2.48 ± 0.7...
-
[98]
The process runs for a maxi- mum of 200 steps or until convergence, using the MSE bidirectional Chamfer distance as the objective function
Alignment: Both the original and rotated samples are aligned to the ground truth target using the Iterative Clos- est Point (ICP) algorithm. The process runs for a maxi- mum of 200 steps or until convergence, using the MSE bidirectional Chamfer distance as the objective function
-
[99]
Chamfer distance performance is evaluated by providing scale, rotation, and translation degrees of freedom to the transformation estimation
Final score: The lower of the two costs from the align- ment steps is reported as the final score. Chamfer distance performance is evaluated by providing scale, rotation, and translation degrees of freedom to the transformation estimation. Model-view chamfer distance is evalua...
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.