Pith. sign in

REVIEW 1 major objections 4 minor 75 references

Unsupervised Monocular 3D Keypoint Discovery from Multi-View Diffusion Priors

T0 review · 1 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read KeyDiff3D predicts 3D keypoints from a single image without 3D labels or calibrated multi-view data, using diffusion features.

desk verdict KeyDiff3D is a genuinely new combination of multi-view diffusion priors with unsupervised monocular 3D keypoints; the central claim is undercut by ground-truth-keypoint preprocessing on Human3.6M, but the idea and the animal/out-of-domain results warrant serious review. read the letter →

arxiv 2507.12336 v2 pith:QPSZ2AY3 submitted 2025-07-16 cs.CV

Subin Jeon , In Cho , Junyoung Hong , Woong Oh Cho , Seon Joo Kim This is my paper · ORCID
classification cs.CV
keywords unsupervisedkeypointdiscovery3Destimationmonocularposemulti-viewdiffusionmodelfeaturesvolumetricunprojectionsingle-viewreconstructionKeyDiff3D
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

KeyDiff3D claims that accurate 3D keypoints can be predicted from a single image, without manual 3D annotations, known camera parameters, or multi-view training footage, by borrowing geometry from a pretrained multi-view diffusion model. The diffusion model supplies both multi-view supervision images and internal features that are lifted into a 3D volume; a self-supervised reconstruction objective then teaches the keypoints. On Human3.6M the method reports 121.34 mm MPJPE with a two-layer regressor, beating unsupervised single-view baselines and roughly matching multi-view methods, and it transfers to birds, dogs, and out-of-domain images. If correct, this removes the main data bottleneck for 3D keypoint learning, making arbitrary object categories tractable from ordinary photos.

What carries the argument

The central mechanism is the multi-view diffusion model used as a 3D feature extractor and supervision source. The paper aggregates decoder features from several U-Net layers at an intermediate denoising timestep with learned scalar weights, unprojects the aggregated 2D features into an $M\times M\times M$ voxel grid using the known projection matrices $P_k$ of the generated views, and fuses them with softmax attention along the view dimension; a 3D CNN then predicts softmax-integrated keypoint coordinates. The same generated views and camera matrices provide the reconstruction loss that makes the keypoints learn without annotations.

What would settle it

Take an input image for which the multi-view diffusion model is known to produce inconsistent views (for example, a left-right flip of the subject or a missing occluded limb) and check whether the predicted 3D keypoints depart from ground truth exactly in the regions the generator failed; the paper's own ablation shows that one degree of camera noise raises MPJPE from 121 mm to 134 mm, so a dataset of such failures should produce a similarly sharp error jump.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that a fixed multi-view diffusion generator can act as a teacher for monocular 3D keypoint discovery. From an input image, the diffusion model is run partway through denoising; its multi-layer decoder features are aggregated, unprojected into a canonical 3D voxel grid using the camera matrices of generated views, and turned into per-keypoint heatmaps via a 3D CNN and integral regression. The same generated views act as reconstruction targets: predicted 3D keypoints are projected onto each view, rendered as Gaussian edge maps, and used to reconstruct the view from the input appearance, with VGG perceptual and mask losses. The resulting single-image predictor attains MPJPE 121.34 mm and P-MPJPE 85.26 mm on Human3.6M, outperforming all compared unsupervised single-view methods and the two-view variant of BKinD-3D, while also producing structurally coherent 2D projections on CUB and Stanford Dogs and enabling skeleton-driven animation of reconstructed 3D objects.

Load-bearing premise

The entire method inherits its geometry from the pretrained multi-view diffusion model: if the generated novel views are not consistent with the camera poses supplied to the generator, or if the generator flips or omits parts of the subject, the unprojection and reconstruction training cannot recover correct 3D keypoints.

Editorial extensions

If this is right

  • Single-image 3D keypoint estimation no longer needs calibrated multi-view capture or 3D labels; unconstrained photos suffice for training.
  • The same pipeline extends across categories (humans, birds, dogs) and transfers to in-the-wild and out-of-domain inputs, so the approach can scale to rare or unannotated object classes.
  • Because the keypoints live in the diffusion model's coordinate frame, they can be used to rig and animate 3D reconstructions from a single image without manual skeleton design.
  • Diffusion-based features beat 2D foundation backbones and 2D-detection-plus-triangulation, indicating that explicit volumetric lifting of diffusion features is what carries the accuracy.
  • Accuracy improves as the number of generated views grows, saturating around four views; even one extra view gives a large jump, so the geometric signal comes from multi-view consistency, not the input image alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If this holds across many object classes, unsupervised keypoint discovery could become the default way to generate 3D pose pseudo-labels, because the only required asset is a pretrained multi-view generator rather than a capture rig.
  • The method's failure modes are inherited from the generator, so KeyDiff3D could double as a diagnostic for multi-view consistency: keypoint error should localize where the diffusion model's novel views are inconsistent.
  • A natural extension is to condition the diffusion features on task-specific prompts or category templates to encourage semantic keypoint correspondences across species.
  • The manipulation pipeline suggests that any 3D-consistent generative prior could be made controllable by lifting its internal features in the same way, not just multi-view diffusion models.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 4 minor

Summary. The paper introduces KeyDiff3D, an unsupervised framework for monocular 3D keypoint estimation that exploits the geometric priors of a pretrained multi-view diffusion model (SV3D). Given a single image, the method runs partial denoising to obtain multi-view features, aggregates them across layers and views, unprojects them into a voxel volume using the diffusion model's camera matrices, and predicts 3D keypoints via soft-argmax on volumetric heatmaps. Training is self-supervised by reconstructing diffusion-generated views from projected keypoint edge maps and a mask loss, with no ground-truth 3D pose used during learning. Experiments on Human3.6M report MPJPE 121.34 mm with a 2-layer MLP regressor, outperforming unsupervised single-view baselines and roughly matching multi-view methods; qualitative results on CUB, Stanford Dogs, DAVIS, GSO, and AP-10K illustrate generalization. The paper also shows a pipeline for animating reconstructed 3D objects using the predicted keypoints and connectivity graph.

Significance. If the annotation-free claim is substantiated, this is a meaningful advance: it replaces expensive multi-view capture and 3D annotation with a pretrained multi-view diffusion prior, enabling single-image 3D keypoint discovery for arbitrary categories. The paper is careful in several respects: the ablations in Table 3 and Table 5 support the key design choices (diffusion features over CLIP/DINOv2/ResNet, volumetric lifting over triangulation, more virtual views), the failure cases in Appendix B are honestly discussed, and Table 6 provides a computational cost analysis. The main caveat is that the central numerical claim on Human3.6M currently depends on ground-truth keypoint-derived preprocessing, so the 'no manual annotations' claim is not yet demonstrated on the primary benchmark.

major comments (1)
  1. [Appendix D.3 / Section 4.1] On Human3.6M, the full pipeline uses ground-truth 2D keypoint locations to prompt SAM and to compute the tight bounding box used for cropping and alignment (Appendix D.3). These are exactly the manual annotations that the Abstract and Section 1 claim to avoid. The ground-truth-derived masks enter the training objective through the mask loss in Eq. (8), and the ground-truth-derived crop changes the input distribution for both SV3D and the keypoint head. Because the headline numbers in Table 1 (MPJPE 121.34 mm) are obtained under this protocol, the central 'unsupervised / no manual annotations' claim is not established for Human3.6M. The fix is concrete and within scope: replace the ground-truth prompts with an automatic person-segmentation prompt (e.g., Grounded-SAM2 with a 'person' text prompt, as already used for animals), and report the resulting Human3.6M metrics. This also ensures the comparison with single-view baselines is fair, since those baselines do not receive keypoint-derived cropping.
minor comments (4)
  1. [Section 4.1 / Eqs. (3), (6)] The statement 'without using camera parameters' is potentially misleading: Eqs. (3) and (6) rely on the fixed camera matrices P_k provided by SV3D. Please clarify that these are the diffusion model's default viewpoint parameters, not per-input calibrated cameras from the dataset, so that the claim is precise.
  2. [Table 1] The rows marked with '*' use a simplified six-action subset; please specify exactly which actions are included and confirm that the comparison with Yang et al.* uses the same subjects, actions, and evaluation protocol so that the comparison is apples-to-apples.
  3. [Table 2 / Appendix A] The large discrepancies between originally reported and reproduced CUB-aligned scores (e.g., Lorenz et al. 3.91 vs 10.34, GANSeg 3.23 vs 15.73) are important context; this information should be summarized in the main text or at least in the table caption, and it should be stated clearly which numbers are from the original papers and which are reproduced by the authors.
  4. [Section 4.2] The reported MPJPE/P-MPJPE values are obtained by fitting a linear or 2-layer MLP regressor from the predicted keypoints to ground-truth 3D joints on the training split. This is the standard protocol in prior unsupervised keypoint discovery works, but the paper should state explicitly that these numbers measure the readability of the discovered representation, not raw keypoint localization error; a complementary correspondence-based evaluation (e.g., nearest-neighbor matching after canonicalization) would strengthen the claim that the predicted keypoints themselves are accurate.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant derivation circularity; the H3.6M GT-keypoint-based SAM/crop preprocessing in Appendix D.3 is a real annotation-purity caveat, not a circular step.

full rationale

The central training loop is not circular: 3D keypoints are predicted from SV3D intermediate features via unprojection (Eqs. 3-4) and supervised by reconstructing SV3D-generated views (Eqs. 7-8), with the binary foreground mask coming from the generated images. Ground-truth 3D pose is never a training input; final MPJPE is obtained through a post-hoc regressor fitted to held-out Human3.6M subjects, the same external benchmark protocol used by prior unsupervised keypoint works. SV3D is an external pretrained model, so invoking its geometry is a dependency, not a redefinition. The only self-citation ([25], on video prediction) is in related work and is not load-bearing. Two manuscript passages deserve explicit flags but do not amount to circularity: Appendix B candidly states failure modes (left-right ambiguity; SV3D view flipping and occlusion failures), and Appendix D.3 reveals that Human3.6M preprocessing uses GT 2D keypoints as SAM prompts and for crop alignment. This means the 'no manual annotations' claim on the main benchmark is not fully demonstrated, and baseline comparisons may be affected; however, no equation or fitted quantity reduces to its own target by construction, so this is an annotation-purity and correctness caveat rather than a circular derivation.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The method's novelty budget is surprisingly lean: it relies on SV3D's pretrained priors plus about six hand-set hyperparameters. The target metric (MPJPE) is not directly optimized during keypoint learning, but tau, K, and N were selected with ablations on the Human3.6M benchmark, so the ledger reflects mild tuning on the evaluation dataset. No new physical or representational entities are introduced beyond standard 3D feature volumes, keypoint heatmaps, and Gaussian splatting.

free parameters (6)
  • keypoint count N = 18 (default; 16/32/48 ablated)
    Hand-chosen balance of interpretability and accuracy; Table 5(a) shows performance improves with more keypoints, but 18 is used as default.
  • diffusion feature timestep tau = 500
    Selected on Human3.6M among 300, 500, 700, 900 (Table 5(b)); mild tuning on the evaluation metric, though scores are close across timesteps.
  • number of views K = 4 (input plus 3 generated)
    Selected among 1-5 (Table 3(c)); performance saturates beyond K=3-4, so this is a robustness-tested choice.
  • voxel grid resolution M = 72
    Implementation choice for memory and accuracy; no ablation reported.
  • loss weights lambda_vgg, lambda_mask, lambda_vic = 1.0, 0.5, 0.1
    Set by hand; no sensitivity analysis reported.
  • Gaussian line width sigma for edge maps = unspecified
    Used in the differentiable line rendering L_ij(u); the value is not reported, and it directly shapes the reconstruction supervision signal.
assumptions (5)
  • domain assumption SV3D's generated views are geometrically consistent with each other and with the input view.
    The entire self-supervision and unprojection pipeline (Section 3.2, Eq. 3-6) assumes the diffusion model's multi-view outputs share one coherent 3D scene; Appendix B documents failures such as subject flipping.
  • domain assumption SV3D's camera matrices P_k are accurate and lie in a fixed canonical frame.
    Unprojection of features into the 3D volume (Eq. 3-4) uses P_k directly; Table 5(e) shows degradation under simulated camera noise above 0.5 degrees.
  • domain assumption Diffusion decoder features at timestep 500 encode pose-relevant 3D structure.
    The feature extraction (Section 3.2) assumes intermediate U-Net features, aggregated with learned weights, contain recoverable 3D geometric cues; supported indirectly by Table 3(a) comparisons against CLIP, DINOv2, and ResNet50.
  • domain assumption The reconstruction objective with affine-augmented appearance does not collapse to appearance copying.
    The training signal (Eq. 7-8) must force keypoints to carry structure; the affine augmentation is introduced specifically because the naive objective permits trivial solutions (Appendix C, Table 5(d)).
  • domain assumption Human3.6M preprocessing with ground-truth 2D keypoints as SAM prompts does not undermine the unsupervised claim.
    Appendix D.3 uses dataset 2D keypoint locations to extract foreground masks for Human3.6M; the abstract and intro claim 'no manual annotations', so this assumption is a partial qualification of that claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unsupervised Monocular 3D Keypoint Discovery from Multi-View Diffusion Priors." pith.science (2026). https://pith.science/paper/QPSZ2AY3

@misc{pith2026250712336,
  author       = {Pith},
  title        = {Pith review of: Unsupervised Monocular 3D Keypoint Discovery from Multi-View Diffusion Priors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QPSZ2AY3}},
  note         = {Machine review of arXiv:2507.12336}
}
read the original abstract

Most existing 3D keypoint estimation methods rely on manual annotations or calibrated multi-view images, both of which are expensive to collect. This paper introduces KeyDiff3D, a framework that can accurately predict 3D keypoints from a single image, thus eliminating the need for such expensive data acquisitions. To achieve this, we leverage powerful geometric priors embedded in a pretrained multi-view diffusion model. In our framework, the diffusion model generates multi-view images from a single image, serving as supervision signals to provide 3D geometric cues to our model. We also introduce a 3D feature extractor that transforms implicit 3D priors embedded in the diffusion features into explicit 3D feature volumes. Beyond accurate keypoint estimation, we further introduce a pipeline that enables manipulation of 3D objects generated by the diffusion model. Experimental results on diverse datasets, including Human3.6M, CUB-200-2011, Stanford Dogs, and several in-the-wild and out-of-domain inputs, highlight the effectiveness of our method in terms of accuracy, generalization, and its ability to enable manipulation of 3D objects generated by the diffusion model from a single image.

Figures

Figures reproduced from arXiv: 2507.12336 by the authors.

Figure 1
Figure 1. KeyDiff3D enables 3D keypoint prediction and object manipulation from a single image using multi-view diffusion priors. It generalizes effectively to in-the-wild and out-of-domain scenarios across diverse categories, including both human and animal domains. Abstract Most existing 3D keypoint estimation methods rely on man￾ual annotations or calibrated multi-view images, both of which are expensive to collect. This p… view at source ↗
Figure 2
Figure 2. The overall pipeline of KeyDiff3D. From a single image, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Qualitative comparison on the Human3.6M dataset. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Qualitative results on the (a) CUB-200-2011 and (b) Stanford Dogs datasets. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Out-of-domain generalization results. (a) In-the-wild DAVIS results and (b) out-of-domain GSO results using a model trained on [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Animatable 3D model results [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison with unsupervised 2D keypoint [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Failure cases. herent cue to distinguish between the left and right sides [12, 56, 68]. Although our network produces consistent keypoint predictions across input and diffusion-generated views, our model has difficulty resolving the left-right am￾biguity present in the…
Figure 9
Figure 9. Figure 9: Keypoint prediction results according to the number of keypoints. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Additional results on Human3.6M dataset. [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Results of our model trained with the Human3.6M dataset, which is tested on (a) in-the-wild and (b) out-of-domain images. [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Additional results on CUB-200-2011 dataset (CUB-align) [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Additional results on Stanford Dogs dataset. [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]
Figure 14
Figure 14. Figure 14: Additional results on the AP-10K dataset using a model trained on Stanford Dogs. [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

75 extracted references · 70 canonical work pages

  1. [1]

    Label-efficient se- mantic segmentation with diffusion models

    Dmitry Baranchuk, Ivan Rubachev, Andrey V oynov, Valentin Khrulkov, and Artem Babenko. Label-efficient se- mantic segmentation with diffusion models. InICLR, 2022. 3

    work page 2022

  2. [2]

    unsupervised-disentagling.https : / / github

    Sandro Braun. unsupervised-disentagling.https : / / github . com / theRealSuperMario / unsupervised - disentangling / tree / reproducing_baselines. 12

    work page

  3. [3]

    3d pictorial structures for multiple view articulated pose es- timation

    Magnus Burenius, Josephine Sullivan, and Stefan Carlsson. 3d pictorial structures for multiple view articulated pose es- timation. InCVPR, pages 3618–3625, 2013. 2

    work page 2013

  4. [4]

    Unsupervised 3d pose estimation with geometric self- supervision

    Ching-Hang Chen, Ambrish Tyagi, Amit Agrawal, Dy- lan Drover, Rohith Mv, Stefan Stojanov, and James M Rehg. Unsupervised 3d pose estimation with geometric self- supervision. InCVPR, pages 5714–5724, 2019. 2

    work page 2019

  5. [5]

    Unsupervised part discovery from con- trastive reconstruction.NeurIPS, 34:28104–28118, 2021

    Subhabrata Choudhury, Iro Laina, Christian Rupprecht, and Andrea Vedaldi. Unsupervised part discovery from con- trastive reconstruction.NeurIPS, 34:28104–28118, 2021. 5, 6

    work page 2021

  6. [6]

    Google scanned objects: A high- quality dataset of 3d scanned household items

    Laura Downs, Anthony Francis, Nate Koenig, Brandon Kin- man, Ryan Hickman, Krista Reymann, Thomas B McHugh, and Vincent Vanhoucke. Google scanned objects: A high- quality dataset of 3d scanned household items. InICRA, pages 2553–2560. IEEE, 2022. 5

    work page 2022

  7. [7]

    Can 3d pose be learned from 2d projections alone? InECCV, pages 78–94, 2018

    David Drover, Graham Taylor, Francis Williams, Derek Nowrouzezahrai, and Surya Prakalp. Can 3d pose be learned from 2d projections alone? InECCV, pages 78–94, 2018. 2

    work page 2018

  8. [8]

    Gaussian frosting: Ed- itable complex radiance fields with real-time rendering

    Antoine Gu ´edon and Vincent Lepetit. Gaussian frosting: Ed- itable complex radiance fields with real-time rendering. In ECCV, pages 413–430. Springer, 2024. 5, 7

    work page 2024

Show all 75 references
  1. [9]

    Deepfly3d, a deep learning-based approach for 3d limb and appendage tracking in tethered, adultDrosophila.eLife, 8:e48571, 2019

    Semih G ¨unel, Helge Rhodin, Daniel Morales, Jo˜ao Campag- nolo, Pavan Ramdya, and Pascal Fua. Deepfly3d, a deep learning-based approach for 3d limb and appendage tracking in tethered, adultDrosophila.eLife, 8:e48571, 2019. 2

    2019

  2. [10]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InCVPR, pages 770–778, 2016. 8

    2016

  3. [11]

    Laten- tkeypointgan: Controlling gans via latent keypoints.arXiv preprint arXiv:2103.15812, 2021

    Xingzhe He, Bastian Wandt, and Helge Rhodin. Laten- tkeypointgan: Controlling gans via latent keypoints.arXiv preprint arXiv:2103.15812, 2021. 6

    2021 arXiv

  4. [12]

    Autolink: Self-supervised learning of human skeletons and object out- lines by linking keypoints.NeurIPS, 35:36123–36141, 2022

    Xingzhe He, Bastian Wandt, and Helge Rhodin. Autolink: Self-supervised learning of human skeletons and object out- lines by linking keypoints.NeurIPS, 35:36123–36141, 2022. 2, 4, 6, 12, 13

    2022

  5. [13]

    Ganseg: Learning to segment by unsupervised hierarchical image generation

    Xingzhe He, Bastian Wandt, and Helge Rhodin. Ganseg: Learning to segment by unsupervised hierarchical image generation. InCVPR, pages 1225–1235, 2022. 6, 12

    2022

  6. [14]

    Unsupervised semantic correspondence using stable diffu- sion.NeurIPS, 36:8266–8279, 2023

    Eric Hedlin, Gopal Sharma, Shweta Mahajan, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. Unsupervised semantic correspondence using stable diffu- sion.NeurIPS, 36:8266–8279, 2023. 3

    2023

  7. [15]

    Unsupervised keypoints from pretrained diffusion models

    Eric Hedlin, Gopal Sharma, Shweta Mahajan, Xingzhe He, Hossam Isack, Abhishek Kar, Helge Rhodin, Andrea Tagliasacchi, and Kwang Moo Yi. Unsupervised keypoints from pretrained diffusion models. InCVPR, pages 22820– 22830, 2024. 2, 3, 6, 12

    2024

  8. [16]

    Unsupervised 3d keypoint discovery with multi-view geom- etry

    Sina Honari, Chen Zhao, Mathieu Salzmann, and Pascal Fua. Unsupervised 3d keypoint discovery with multi-view geom- etry. In3DV, pages 1584–1593. IEEE, 2024. 1, 2, 5, 6

    2024

  9. [17]

    Scops: Self-supervised co-part segmentation

    Wei-Chih Hung, Varun Jampani, Sifei Liu, Pavlo Molchanov, Ming-Hsuan Yang, and Jan Kautz. Scops: Self-supervised co-part segmentation. InCVPR, 2019. 6

    2019

  10. [18]

    Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3. 6m: Large scale datasets and pre- dictive methods for 3d human sensing in natural environ- ments.IEEE TPAMI, 36(7):1325–1339, 2013. 2, 5

    2013

  11. [19]

    Weakly- supervised 3d human pose learning via multi-view images in the wild

    Umar Iqbal, Pavlo Molchanov, and Jan Kautz. Weakly- supervised 3d human pose learning via multi-view images in the wild. InCVPR, pages 5243–5252, 2020. 2

    2020

  12. [20]

    Learnable triangulation of human pose

    Karim Iskakov, Egor Burkov, Victor Lempitsky, and Yury Malkov. Learnable triangulation of human pose. InICCV,

  13. [21]

    Unsupervised learning of object landmarks through conditional image generation.NeurIPS, 31, 2018

    Tomas Jakab, Ankush Gupta, Hakan Bilen, and Andrea Vedaldi. Unsupervised learning of object landmarks through conditional image generation.NeurIPS, 31, 2018. 2, 5

    2018

  14. [22]

    Rupp, Evyn S

    Pierre Karashchuk, Katie L. Rupp, Evyn S. Dickinson, Sarah Walling-Bell, Elischa Sanders, Eiman Azim, Bingni W. Brunton, and John C. Tuthill. Anipose: A toolkit for robust markerless 3d pose estimation.Cell Reports, 36(13):109730,

  15. [23]

    3d gaussian splatting for real-time radiance field rendering.ACM TOG, 42(4), 2023

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM TOG, 42(4), 2023. 5, 7

    2023

  16. [24]

    Novel dataset for fine-grained image categorization

    Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. Novel dataset for fine-grained image categorization. InCVPR, Colorado Springs, CO, 2011. 2, 5, 6

    2011

  17. [25]

    Unsupervised keypoint learning for guiding class- conditional video prediction.NeurIPS, 32, 2019

    Yunji Kim, Seonghyeon Nam, In Cho, and Seon Joo Kim. Unsupervised keypoint learning for guiding class- conditional video prediction.NeurIPS, 32, 2019. 2

    2019

  18. [26]

    Self- supervised learning of 3d human pose using multi-view ge- ometry

    Muhammed Kocabas, Salih Karagoz, and Emre Akbas. Self- supervised learning of 3d human pose using multi-view ge- ometry. InCVPR, pages 1077–1086, 2019. 2, 5

    2019

  19. [27]

    Eschernet: A generative model for scalable view synthesis

    Xin Kong, Shikun Liu, Xiaoyang Lyu, Marwan Taher, Xi- aojuan Qi, and Andrew J Davison. Eschernet: A generative model for scalable view synthesis. InCVPR, pages 9503– 9513, 2024. 3

    2024

  20. [28]

    Unsupervised learning of object keypoints for perception and control.NeurIPS, 32, 2019

    Tejas D Kulkarni, Ankush Gupta, Catalin Ionescu, Sebas- tian Borgeaud, Malcolm Reynolds, Andrew Zisserman, and V olodymyr Mnih. Unsupervised learning of object keypoints for perception and control.NeurIPS, 32, 2019. 2

    2019

  21. [29]

    Self-supervised 3d human pose estimation via part guided novel image synthesis

    Jogendra Nath Kundu, Mohak Gor, Arunava Agarwala, C V Jawahar, and Kiran V Krishna. Self-supervised 3d human pose estimation via part guided novel image synthesis. In CVPR, pages 6152–6162, 2020. 2, 6

    2020

  22. [30]

    Kinematic-structure-preserved representation for unsuper- vised 3d human pose estimation

    Jogendra Nath Kundu, Siddharth Seth, Mugalodi Rakesh, Venkatesh Babu Radhakrishnan, Anirban Chakraborty, et al. Kinematic-structure-preserved representation for unsuper- vised 3d human pose estimation. InAAAI, pages 11312– 11319, 2020. 2, 6

    2020

  23. [31]

    3d human pose estimation from monocular images with deep convolutional neural net- work

    Sijin Li and Antoni B Chan. 3d human pose estimation from monocular images with deep convolutional neural net- work. InAsian conference on computer vision, pages 332–

  24. [32]

    Geometry-driven self- supervised method for 3d human pose estimation

    Yang Li, Kan Li, Shuai Jiang, Ziyue Zhang, Congzhen- tao Huang, and Richard Yi Da Xu. Geometry-driven self- supervised method for 3d human pose estimation. InAAAI, pages 11442–11449, 2020. 2

    2020

  25. [33]

    Multiple view geometry transformers for 3d human pose estimation

    Ziwei Liao, Jialiang Zhu, Chunyu Wang, Han Hu, and Steven L Waslander. Multiple view geometry transformers for 3d human pose estimation. InCVPR, pages 708–717,

  26. [34]

    Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age. InICLR, 2024. 3

    2024

  27. [35]

    Unsupervised part-based disentangling of object shape and appearance

    Dominik Lorenz, Leonard Bereska, Timo Milbich, and Bjorn Ommer. Unsupervised part-based disentangling of object shape and appearance. InCVPR, pages 10955–10964, 2019. 2, 5, 6, 12

    2019

  28. [36]

    Diffusion hyperfeatures: Search- ing through time and space for semantic correspondence

    Grace Luo, Lisa Dunlap, Dong Huk Park, Aleksander Holyn- ski, and Trevor Darrell. Diffusion hyperfeatures: Search- ing through time and space for semantic correspondence. NeurIPS, 36:47500–47510, 2023. 3

    2023

  29. [37]

    A simple yet effective baseline for 3d human pose estimation

    Julieta Martinez, Rayat Hossain, Javier Romero, and James J Little. A simple yet effective baseline for 3d human pose estimation. InICCV, pages 2640–2649, 2017. 2

    2017

  30. [38]

    Unsupervised learning of object structure and dynamics from videos

    Matthias Minderer, Chen Sun, Ruben Villegas, Forrester Cole, Kevin P Murphy, and Honglak Lee. Unsupervised learning of object structure and dynamics from videos. NeurIPS, 32, 2019. 2

    2019

  31. [39]

    V2v-posenet: V oxel-to-voxel prediction network for accu- rate 3d hand and human pose estimation from a single depth map

    Gyeongsik Moon, Ju Yong Chang, and Kyoung Mu Lee. V2v-posenet: V oxel-to-voxel prediction network for accu- rate 3d hand and human pose estimation from a single depth map. InCVPR, pages 5079–5088, 2018. 15

    2018

  32. [40]

    Emerdiff: Emerging pixel-level semantic knowledge in diffusion models

    Koichi Namekata, Amirmojtaba Sabour, Sanja Fidler, and Seung Wook Kim. Emerdiff: Emerging pixel-level semantic knowledge in diffusion models. InICLR, 2024. 3

    2024

  33. [41]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 8

    2023 arXiv

  34. [42]

    3d human pose estimation using convolutional neural networks with 2d pose information

    Sungheon Park, Jihye Hwang, and Nojun Kwak. 3d human pose estimation using convolutional neural networks with 2d pose information. InECCV Workshop, pages 156–169. Springer, 2016. 2

    2016

  35. [43]

    Harvesting multiple views for marker-less 3d human pose annotations

    Georgios Pavlakos, Xiaowei Zhou, Konstantinos G Derpa- nis, and Kostas Daniilidis. Harvesting multiple views for marker-less 3d human pose annotations. InCVPR, pages 6988–6997, 2017. 2

    2017

  36. [44]

    Perazzi, J

    F. Perazzi, J. Pont-Tuset, B. McWilliams, L. Van Gool, M. Gross, and A. Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In CVPR, 2016. 5

    2016

  37. [45]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. InICML, pages 8748–8763. PmLR, 2021. 8

    2021

  38. [46]

    Grounded sam: Assembling open-world models for diverse visual tasks,

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kun- chang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks,

  39. [47]

    Black, and Silvia Zuffi

    Nadine R ¨uegg, Shashank Tripathi, Konrad Schindler, Michael J. Black, and Silvia Zuffi. BITE: Beyond priors for improved three-D dog pose estimation. InCVPR, pages 8867–8876, 2023. 2

    2023

  40. [48]

    3d human pose estimation: A review of the literature and analysis of covariates.Computer Vision and Image Understanding, 152:1–20, 2016

    Nikolaos Sarafianos, Bogdan Boteanu, Bogdan Ionescu, and Ioannis A Kakadiaris. 3d human pose estimation: A review of the literature and analysis of covariates.Computer Vision and Image Understanding, 152:1–20, 2016. 2

    2016

  41. [49]

    Animating arbitrary objects via deep motion transfer

    Aliaksandr Siarohin, St ´ephane Lathuili`ere, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. Animating arbitrary objects via deep motion transfer. InCVPR, pages 2377–2386, 2019. 2

    2019

  42. [50]

    First order motion model for image animation.NeurIPS, 32, 2019

    Aliaksandr Siarohin, St ´ephane Lathuili`ere, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. First order motion model for image animation.NeurIPS, 32, 2019

    2019

  43. [51]

    Motion- supervised co-part segmentation

    Aliaksandr Siarohin, Subhankar Roy, St ´ephane Lathuili `ere, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. Motion- supervised co-part segmentation. InICPR, pages 9650–

  44. [52]

    Self-supervised 3d human pose estimation from a single image

    Jose Sosa and David Hogg. Self-supervised 3d human pose estimation from a single image. InCVPR, pages 4788–4797,

  45. [53]

    Self-supervised keypoint discovery in behavioral videos

    Jennifer J Sun, Serim Ryou, Roni H Goldshmid, Bran- don Weissbourd, John O Dabiri, David J Anderson, Ann Kennedy, Yisong Yue, and Pietro Perona. Self-supervised keypoint discovery in behavioral videos. InCVPR, pages 2171–2180, 2022. 2

    2022

  46. [54]

    Bkind-3d: Self-supervised 3d keypoint discovery from multi-view videos

    Jennifer J Sun, Lili Karashchuk, Amil Dravid, Serim Ryou, Sonia Fereidooni, John C Tuthill, Aggelos Katsaggelos, Bingni W Brunton, Georgia Gkioxari, Ann Kennedy, et al. Bkind-3d: Self-supervised 3d keypoint discovery from multi-view videos. InCVPR, pages 9001–9010, 2023. 1, 2, 4, 5, 6

    2023

  47. [55]

    Integral human pose regression

    Xiao Sun, Bin Xiao, Fangyin Wei, Shuang Liang, and Yichen Wei. Integral human pose regression. InECCV, pages 529– 545, 2018. 4

    2018

  48. [56]

    Discovery of latent 3d key- points via end-to-end geometric reasoning.NeurIPS, 31,

    Supasorn Suwajanakorn, Noah Snavely, Jonathan J Tomp- son, and Mohammad Norouzi. Discovery of latent 3d key- points via end-to-end geometric reasoning.NeurIPS, 31,

  49. [57]

    Emergent correspondence from image diffusion.NeurIPS, 36:1363–1389, 2023

    Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffusion.NeurIPS, 36:1363–1389, 2023. 3

    2023

  50. [58]

    Unsu- pervised learning of object landmarks by factorized spatial embeddings

    James Thewlis, Hakan Bilen, and Andrea Vedaldi. Unsu- pervised learning of object landmarks by factorized spatial embeddings. InICCV, 2017. 2, 6

    2017

  51. [59]

    Metapose: Fast 3d pose from multiple views without 3d supervision

    Ben Usman, Andrea Tagliasacchi, Kate Saenko, and Avneesh Sud. Metapose: Fast 3d pose from multiple views without 3d supervision. InCVPR, pages 6759–6770, 2022. 2

    2022

  52. [60]

    Sv3d: Novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion

    Vikram V oleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion. InECCV, pages 439–457. Springer, 2...

    2024

  53. [61]

    The caltech-ucsd birds-200-2011 dataset

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Per- ona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011. 2, 5, 6

    2011

  54. [62]

    Repnet: Weakly super- vised training of an adversarial reprojection network for 3d human pose estimation

    Bastian Wandt and Bodo Rosenhahn. Repnet: Weakly super- vised training of an adversarial reprojection network for 3d human pose estimation. InCVPR, pages 7782–7791, 2019. 2

    2019

  55. [63]

    Canonpose: Self-supervised monoc- ular 3d human pose estimation in the wild

    Bastian Wandt, Marco Rudolph, Petrissa Zell, Helge Rhodin, and Bodo Rosenhahn. Canonpose: Self-supervised monoc- ular 3d human pose estimation in the wild. InCVPR, pages 13294–13304, 2021. 2

    2021

  56. [64]

    Ele- pose: Unsupervised 3d human pose estimation by predict- ing camera elevation and learning normalizing flows on 2d poses

    Bastian Wandt, Minh Nguyen, and Bodo Rosenhahn. Ele- pose: Unsupervised 3d human pose estimation by predict- ing camera elevation and learning normalizing flows on 2d poses. InCVPR, pages 10411–10420, 2022. 2

    2022

  57. [65]

    Imagedream: Image-prompt multi-view diffusion for 3d generation.arXiv preprint arXiv:2312.02201, 2023

    Peng Wang and Yichun Shi. Imagedream: Image-prompt multi-view diffusion for 3d generation.arXiv preprint arXiv:2312.02201, 2023. 3

    2023 arXiv

  58. [66]

    Open-vocabulary panop- tic segmentation with text-to-image diffusion models

    Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiao- long Wang, and Shalini De Mello. Open-vocabulary panop- tic segmentation with text-to-image diffusion models. In CVPR, pages 2955–2966, 2023. 3

    2023

  59. [67]

    Diffusion model as repre- sentation learner

    Xingyi Yang and Xinchao Wang. Diffusion model as repre- sentation learner. InICCV, pages 18938–18949, 2023. 3

    2023

  60. [68]

    Mask-as-supervision: Lever- aging unified mask information for unsupervised 3d pose es- timation

    Yuxuan Yang, Chen Qian, Jiefeng Li, Xiao Sun, Bin Xiao, Yichen Wei, and Limin Wang. Mask-as-supervision: Lever- aging unified mask information for unsupervised 3d pose es- timation. InECCV, 2024. 2, 6, 13

    2024

  61. [69]

    Ap-10k: A benchmark for animal pose esti- mation in the wild

    Hang Yu, Yufei Xu, Jing Zhang, Wei Zhao, Ziyu Guan, and Dacheng Tao. Ap-10k: A benchmark for animal pose esti- mation in the wild. InNeurIPS Workshop, 2021. 5

    2021

  62. [70]

    Exploring diffusion time-steps for unsupervised representation learning

    Zhongqi Yue, Jiankun Wang, Qianru Sun, Lei Ji, Eric I Chang, and Hanwang Zhang. Exploring diffusion time-steps for unsupervised representation learning. InICLR, 2024. 3

    2024

  63. [71]

    A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence.NeurIPS, 36: 45533–45547, 2023

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Pola- nia Cabrera, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence.NeurIPS, 36: 45533–45547, 2023. 3

    2023

  64. [72]

    Unsupervised discovery of object land- marks as structural representations

    Yuting Zhang, Yijie Guo, Yixin Jin, Yijun Luo, Zhiyuan He, and Honglak Lee. Unsupervised discovery of object land- marks as structural representations. InCVPR, pages 2694– 2703, 2018. 2, 6

    2018

  65. [73]

    Triangulation residual loss for data- efficient 3d pose estimation.NeurIPS, 36:12721–12732,

    Jiachen Zhao, Tao Yu, Liang An, Yipeng Huang, Fang Deng, and Qionghai Dai. Triangulation residual loss for data- efficient 3d pose estimation.NeurIPS, 36:12721–12732,

  66. [74]

    Towards 3d human pose estimation in the wild: a weakly-supervised approach

    Xingyi Zhou, Qixing Huang, Xiao Sun, Xiangyang Xue, and Yichen Wei. Towards 3d human pose estimation in the wild: a weakly-supervised approach. InICCV, pages 398–407,

  67. [2017]

    CUB-200-2011 comparison Baseline reproduction.Most unsupervised 2D keypoint methods [12, 13, 15, 35] report their performance on the CUB-align protocol introduced in [35]

    2 A. CUB-200-2011 comparison Baseline reproduction.Most unsupervised 2D keypoint methods [12, 13, 15, 35] report their performance on the CUB-align protocol introduced in [35]. However, we note that it is challenging to ensure a fair and fully controlled comparison under this ...

    2011

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.