Pith. sign in

REVIEW 4 major objections 6 minor 49 references

MetaFE-DE: Learning Meta Feature Embedding for Depth Estimation from Monocular Endoscopic Images

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Diffusion-pretrained meta features, aligned by cross normalization, set a new state of the art in monocular endoscopic depth estimation.

desk verdict Useful empirical recipe for endoscopic depth estimation, but the 'meta feature' claim is over-sold: the fixed cross-normalization is a plausible fusion trick, not a demonstrated modality-independent representation. read the letter →

arxiv 2502.03493 v1 pith:K2VOH25Q submitted 2025-02-05 eess.IV cs.CV

Dawei Lu , Deqiang Xiao , Danni Ai , Jingfan Fan , Tianyu Fu , Yucong Lin , Hong Song , Xujiong Ye
show 2 more authors
Lei Zhang Jian Yang
This is my paper · ORCID
classification eess.IVcs.CV
keywords monoculardepthestimationendoscopicimagingmetafeatureembeddinglatentdiffusionmodelcrossnormalizationself-supervisedlearningtemporalrepresentationmedicalimageanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that features learned by a temporal conditioning diffusion model can be aligned with the current frame's spatial latent features to form a 'meta feature embedding' that represents the physical endoscopic scene independently of RGB or depth modality. The claim is that this embedding, obtained without any depth labels, can be decoded directly into an accurate depth map, and that doing so outperforms prior self-supervised methods and generalizes to a new dataset without fine-tuning. A sympathetic reader should care because it offers a recipe: pretrain generatively on unlabeled endoscopic video, align two latent spaces with a parameter-free cross normalization, then train only a small decoder for the target dense prediction task.

What carries the argument

The central object is the meta feature embedding $Z_t^*$, built by Eq. (5): the current frame's VAE latent $Z_t$ is normalized by the mean and variance of the diffusion forecast $\hat{Z}_t$ and then rescaled by $\gamma$ and added to $\hat{Z}_t$. This parameter-free cross normalization is the mechanism claimed to align the temporal diffusion latent (learned from the three previous frames through a temporal conditioned diffusion model) with the spatial latent of the current frame, producing a single representation that can be decoded into either modality by a VAE-decoder-style network.

What would settle it

Compare depth accuracy when $Z_t$ in Eq. (5) is replaced by the VAE latent of a different frame or by a random latent with the same mean and variance. If the resulting depth maps remain as accurate as those obtained with the true current-frame latent, then the proposed alignment is not what carries the spatial information, and the MetaFE claim would be falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the aligned features $Z_t^*$, obtained by cross-normalizing the temporal diffusion latent $\hat{Z}_t$ with the current frame's VAE latent $Z_t$, constitute a meta feature embedding: a single latent representation from which both an RGB image and a depth image can be decoded. The authors demonstrate this by first pretraining a temporal conditioned latent diffusion model on sequences of three previous frames, then decoding $Z_t^*$ into depth with a self-supervised view-synthesis objective plus brightness calibration. They report that this pipeline reaches an Abs Rel of 0.056 on SCARED versus 0.060 for MonoDiffusion and 0.061 for LiteMono, and that when transferred to Hamlyn without fine-tuning it achieves 0.071 Abs Rel versus 0.089 for MonoDiffusion. They also report CKA feature-similarity evidence that RGB and depth decoders share features in deeper layers, which they read as support for the claim that both modalities are decoded from the same physical-entity representation.

Load-bearing premise

The load-bearing premise is that the cross normalization in Eq. (5) genuinely aligns the diffusion forecast's latent with the current frame's spatial latent, so that the blended $Z_t^*$ carries the current frame's spatial content rather than being a hand-tuned average of two unrelated features.

Editorial extensions

If this is right

  • Depth can be decoded directly from the meta feature, without first reconstructing the RGB image, so the pretraining representation itself carries the information needed for the downstream task.
  • The same aligned feature can be reused for other dense prediction targets (surface normals, segmentation) by replacing only the decoder, since the paper argues RGB and depth share a common abstract feature space.
  • Because the depth decoder performs nearly the same with or without RGB-pretrained weights once cross normalization is used, the alignment step, not the weight initialization, is the component that matters.
  • Using the SCARED-trained model on Hamlyn without fine-tuning still improves over MonoDiffusion, which the paper presents as evidence that the meta features are portable across endoscopic datasets.
  • The two-stage recipe (generative pretraining then self-supervised decoding) works without any ground-truth depth, so it can be applied to surgical video where annotations are scarce.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A decisive extension would be to replace the cross-normalized latent $Z_t^*$ with a simple sum or concatenation of $\hat{Z}_t$ and $Z_t$; if depth accuracy is unchanged, the distribution-alignment story is not the active ingredient.
  • The method's cross-dataset transfer result is its strongest evidence for modality-independent features, but the paper does not report per-sequence or domain-shift analyses; an extension would be to measure how the gap to MonoDiffusion grows with appearance and tissue-type shift.
  • If the MetaFE idea is right, the same temporal-conditioning plus cross-normalization pretraining should also improve monocular depth estimation in other annotation-scarce endoscopic or ultrasound settings with brightness drift.
  • The CKA analysis could be turned into a quantitative criterion: define a threshold of shared-layer similarity above which a new decoder can be expected to benefit from the pretrained meta features, making the representation checkable before training.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces MetaFE-DE, a two-stage self-supervised framework for monocular endoscopic depth estimation. In the first stage, a temporal latent feature is formed from the three previous frames' VAE latents and used to condition a latent diffusion model (TC-DM) that predicts the current frame's latent. This diffusion latent is then combined with the current frame's VAE latent through a fixed cross-normalization operation (Eq. 5), producing the proposed 'meta feature embedding' Z*_t. In the second stage, Z*_t is decoded into depth using a self-supervised photometric loss with a brightness-calibration module. Experiments on SCARED, EndoSLAM, and Hamlyn report consistent improvements over MonoDepth2, LiteMono, and MonoDiffusion, with confidence intervals and an ablation showing that both the cross-normalization and the pretrained-RGB-decoder initialization contribute. The paper also presents CKA-based feature-similarity analysis to argue that RGB and depth decoding share an abstract feature space.

Significance. If the MetaFE claim is accepted, the proposed two-stage pretraining-plus-decoding recipe could transfer to other dense prediction tasks in endoscopic imaging and beyond. The empirical improvements are consistent across three datasets, and the Hamlyn zero-shot transfer (with median scaling) is a meaningful generalization signal. The ablation in Table 4 shows that the cross-normalization step gives a real accuracy gain, which is a useful practical finding. The main weakness is that the evidence for the conceptual claim — that the cross-normalized features constitute a modality-independent physical-entity representation — is indirect: the specific alignment mechanism in Eq. (5) is not isolated from ordinary feature fusion, the statistical significance reporting is incomplete, and the CKA analysis is underspecified and post hoc. These issues do not invalidate the empirical recipe, but they do prevent the conceptual claim from being fully supported.

major comments (4)
  1. [3.1.3, Eq. (3)–(5)] The cross-normalization statistics are defined over the batch dimension (n in Eq. (3)–(4)). Please specify how this operation is applied at inference. If a test batch of size 1 is used, the mean and variance of \hat{Z}_t reduce to that single sample, making the normalization ill-defined (zero variance) and the result highly dependent on batch composition. If the statistics are instead intended over spatial dimensions, the text and equations should be corrected. In addition, the operation is a fixed scalar affine transformation with γ=0.5; it has no channel- or location-specific alignment mechanism. The assertion in Section 3 that cross normalization 'aligns the distributions... while preserving the maximum amount of original information' is therefore unsupported. To attribute the gains in Table 4 to the alignment mechanism rather than to the temporal-diffusion pretraining, add controls that replace Eq. (5) with a learned per-channel affine alignment, a simple concatenation followed by a convolution, or plain averaging, and report the resulting depth metrics.
  2. [4.3.1] The paper states that 'the paired t-test is performed for the statistical significance validation on the improvements of performance' and reports 'p < 0.05', but no p-values, test statistics, or degrees of freedom are provided anywhere in the manuscript or appendix. Please report the actual p-values (or effect sizes) per metric and dataset, and clarify whether the pairing is per image across the test set. Without these details, the statistical significance claim is not verifiable from the manuscript.
  3. [Table 3 and Eq. (18)] The Hamlyn results are described as 'significant improvements without fine-tuning', but the evaluation uses the ground-truth median scaling defined in Eq. (18). This is standard for monocular depth estimation, but the text should state explicitly that scale is calibrated using Hamlyn ground-truth medians; otherwise the 'without fine-tuning' phrasing is misleading. Moreover, because the transfer is a whole-dataset evaluation, it does not isolate the MetaFE construction from the temporal-diffusion pretraining, the decoder architecture, or the brightness-calibration module. A transfer experiment that ablates the cross-normalization step (or replaces it with an alternative fusion) on Hamlyn would strengthen the generalization claim.
  4. [4.3.1, Fig. 5 and Appendix D.3] The text refers to 'CKA (see Appendix D.3)', but Appendix D.3 (titled 'Meta Feature Decoding') does not describe CKA; it only contains a sentence and Fig. 9. Please provide the CKA definition (kernel choice, feature set, how the PCA alignment of different-scale layers is performed) and a proper reference. Also, the interpretation of the CKA results is not compelling as evidence for modality independence: the depth decoder in Fig. 5A is initialized from the RGB decoder, so high similarity in deeper layers may reflect shared initialization rather than a shared representation derived from the cross-normalized features. Please clarify what specific claim the CKA analysis supports and how it distinguishes shared representation from shared initialization.
minor comments (6)
  1. [Abstract] The word 'surffer' should be 'suffer'.
  2. [Tables 1–3] The header 'Sql Rel' should be 'Sq Rel'.
  3. [4.3.1] The dataset name is inconsistent: 'Endo-Slam' appears in the text while 'EndoSlam' is used in Table 2 and Section 4.1; please unify the spelling.
  4. [3.2.3 and Appendix C.1] There is a typo: 'th scale of 256 × 320' should be 'the scale of 256 × 320'.
  5. [5. Discussion] The words 'conject' and 'instant' should be 'conjecture' and 'instance', respectively.
  6. [Fig. 5 and Appendix A.2] The text contains 'docoder' (should be 'decoder'), 'infromation' (should be 'information'), and 'yeilded' (should be 'yielded'), which should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the depth-estimation result is validated against independent ground truth and the cross-normalization is adopted from an external method.

full rationale

No load-bearing circular step can be exhibited. Phase 1 constructs Z*_t via Eq. (5) from the current-frame VAE latent Zt and the temporal diffusion latent \hatZt using cross normalization cited from an external source [13]; Phase 2 trains a depth decoder with photometric loss and reports metrics against held-out ground truth on SCARED, EndoSLAM, and Hamlyn. The Hamlyn result is a direct transfer without fine-tuning, not a fitted prediction. Table 4 ablates cross normalization (Abs Rel 0.060 vs. 0.056), which is an empirical control rather than a reduction of the prediction to the method's inputs. The only definitional aspect is that "meta feature" is defined as the aligned features of Eq. (5) and as being decodable into RGB or depth, so the existence of an embedding with that name is partly established by the definition; however, the quantitative depth-estimation claim does not reduce to that definition. No self-citation chain is load-bearing: the alignment and brightness-calibration components cite ControlNext [13] and AF-Net [14], both outside the present author set. There is no equation in which the predicted depth equals an input parameter by construction, and the post hoc CKA analysis is interpretive rather than part of the derivation. The paper is therefore self-contained against external benchmarks on the core depth-estimation claim.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central claim rests on a chain of unverified modeling assumptions: that diffusion pretraining transfers to depth, that cross normalization preserves meaning, that a linear projection of past frames captures temporal structure, and that CKA similarity implies a physical shared representation. The only fitted constants are hyperparameters plus the ground-truth median scaling used at evaluation.

free parameters (3)
  • gamma (cross-normalization scale) = 0.5
    Scaling parameter in Eq. (5) that controls how strongly the normalized current-frame latent is blended with the diffusion latent; set by hand, not learned or swept.
  • Loss weights lambda_1, lambda_2, lambda_3 = 0.01, 0.01, 0.0001
    Weights in Eq. (12) for the appearance-flow, auxiliary, and edge-smoothness regularization terms in the depth decoding phase; chosen manually and adopted from AF-Net.
  • Per-image median scaling factor = Median(D_gt)/Median(D_pred) per test image
    Applied at inference in Eq. (18) using ground-truth medians before computing depth metrics; a GT-fitted normalization that affects all reported numbers.
assumptions (5)
  • domain assumption Self-supervised photometric warping loss is a sufficient supervisory signal for learning depth from endoscopic videos.
    Adopted from Monodepth2 and AF-Net; the depth decoder is trained to minimize appearance differences between warped and original frames, which assumes Lambertian surfaces and accurate pose.
  • domain assumption Diffusion denoising pretraining with the objective in Eq. (2) learns latent features that transfer to a downstream depth-decoding task.
    The paper relies on prior representation-learning results rather than proving that temporal diffusion features carry depth-relevant information.
  • ad hoc to paper Cross normalization in Eq. (5) aligns the distributions of the temporal diffusion latent and the current-frame VAE latent while preserving semantic content.
    The paper asserts this alignment in Section 3.1.3 without a derivation or independent check; correctness is inferred from downstream depth metrics.
  • ad hoc to paper Concatenating the previous three frames' VAE latents and applying a linear projection adequately encodes temporal dynamics.
    No recurrent or motion model is used; the temporal latent is a learned linear mixture of past frame features, which is an untested modeling assumption.
  • ad hoc to paper High CKA similarity in deeper decoder layers is evidence that RGB and depth decoding share a modality-independent physical representation.
    The interpretation in Section 5 goes beyond what similarity of trained-network features can establish; shared features could simply reflect shared architecture.
invented entities (1)
  • MetaFE (meta feature embedding)
    purpose: A latent representation claimed to encode the physical endoscopic scene independent of RGB or depth modality, decodable into either.
    No external falsifiable handle is provided; the only evidence is that the paper's own decoders produce good depth and that its own feature-similarity analysis shows sharing. The concept is defined by the very ability being tested, so it functions as a postulated entity rather than an independently measured one.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MetaFE-DE: Learning Meta Feature Embedding for Depth Estimation from Monocular Endoscopic Images." pith.science (2026). https://pith.science/paper/K2VOH25Q

@misc{pith2026250203493,
  author       = {Pith},
  title        = {Pith review of: MetaFE-DE: Learning Meta Feature Embedding for Depth Estimation from Monocular Endoscopic Images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K2VOH25Q}},
  note         = {Machine review of arXiv:2502.03493}
}
read the original abstract

Depth estimation from monocular endoscopic images presents significant challenges due to the complexity of endoscopic surgery, such as irregular shapes of human soft tissues, as well as variations in lighting conditions. Existing methods primarily estimate the depth information from RGB images directly, and often surffer the limited interpretability and accuracy. Given that RGB and depth images are two views of the same endoscopic surgery scene, in this paper, we introduce a novel concept referred as ``meta feature embedding (MetaFE)", in which the physical entities (e.g., tissues and surgical instruments) of endoscopic surgery are represented using the shared features that can be alternatively decoded into RGB or depth image. With this concept, we propose a two-stage self-supervised learning paradigm for the monocular endoscopic depth estimation. In the first stage, we propose a temporal representation learner using diffusion models, which are aligned with the spatial information through the cross normalization to construct the MetaFE. In the second stage, self-supervised monocular depth estimation with the brightness calibration is applied to decode the meta features into the depth image. Extensive evaluation on diverse endoscopic datasets demonstrates that our approach outperforms the state-of-the-art method in depth estimation, achieving superior accuracy and generalization. The source code will be publicly available.

Figures

Figures reproduced from arXiv: 2502.03493 by the authors.

Figure 1
Figure 1. This paper proposes the MetaFE that represents physical [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Prior studies [10] suggest that the text and image jointly represent the same entity in the physical world. This paper, how￾ever, categorizes modalities into non-anthropocentric space and anthropocentrically defined space based on their susceptibility to human cognition. For example, the modalities such as RGB and depth image are unaffected by human cognition, thus they are able to reflect the intrinsic physical pro… view at source ↗
Figure 3
Figure 3. The structure of the proposed framework (MetaFE-DE), [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: By decoding the depth information from MetaFE, our [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Feature similarity using CKA, with axes representing [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 8
Figure 8. Figure 8: The generated RGB images with and without depth in [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 6
Figure 6. Figure 6: The network architecture for the method using the depth [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: The network architecture for the method without the [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 9
Figure 9. Figure 9: The meta feature is directly decoded into the depth im [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: More depth estimation examples on SCARED dataset. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 36 canonical work pages

  1. [1]

    Vali- dation of stereo vision based liver surface reconstruction for image guided surgery

    Teatini Andrea, Wang Congcong, Alaya Cheikh Faouzi, Beghdadi Azeddine, Edwin Bjørn, and Elle Ole Jakob. Vali- dation of stereo vision based liver surface reconstruction for image guided surgery. In2018 Colour and Visual Computing Symposium (CVCS), pages 1–6. IEEE, 2018. 1

    work page 2018

  2. [2]

    Monocular real- time hand shape and motion capture using multi-modal data

    Yuxiao Zhou, Marc Habermann, Weipeng Xu, Ikhsanul Habibie, Christian Theobalt, and Feng Xu. Monocular real- time hand shape and motion capture using multi-modal data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5346–5355, 2020. 1

    work page 2020

  3. [3]

    Sparse- then-dense alignment-based 3d map reconstruction method for endoscopic capsule robots

    Mehmet Turan, Yusuf Yigit Pilavci, Ipek Ganiyusufoglu, Helder Araujo, Ender Konukoglu, and Metin Sitti. Sparse- then-dense alignment-based 3d map reconstruction method for endoscopic capsule robots. Machine Vision and Applica- tions, 29:345–359, 2018. 1

    work page 2018

  4. [4]

    Slam-based dense surface reconstruction in monocular minimally invasive surgery and its application to augmented reality

    Long Chen, Wen Tang, Nigel W John, Tao Ruan Wan, and Jian Jun Zhang. Slam-based dense surface reconstruction in monocular minimally invasive surgery and its application to augmented reality. Computer Methods and Programs in Biomedicine, 158:135–146, 2018. 1

    work page 2018

  5. [5]

    Unsupervised odometry and depth learning for endoscopic capsule robots

    Mehmet Turan, Evin Pinar Ornek, Nail Ibrahimli, Can Giracoglu, Yasin Almalioglu, Mehmet Fatih Yanik, and Metin Sitti. Unsupervised odometry and depth learning for endoscopic capsule robots. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1801–1807. IEEE, 2018. 1, 4

    work page 2018

  6. [6]

    Dense depth estimation in monocular endoscopy with self- supervised learning methods

    Xingtong Liu, Ayushi Sinha, Masaru Ishii, Gregory D Hager, Austin Reiter, Russell H Taylor, and Mathias Unberath. Dense depth estimation in monocular endoscopy with self- supervised learning methods. IEEE Transactions on Medical Imaging, 39(5):1438–1447, 2019. 1

    work page 2019

  7. [7]

    Digging into self-supervised monocular depth estimation

    Cl ´ement Godard, Oisin Mac Aodha, Michael Firman, and Gabriel J Brostow. Digging into self-supervised monocular depth estimation. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 3828–3838,

  8. [8]

    Unsupervised learning of depth and ego-motion from video

    Tinghui Zhou, Matthew Brown, Noah Snavely, and David G Lowe. Unsupervised learning of depth and ego-motion from video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1851–1858, 2017. 1, 4, 13

    work page 2017

Show all 49 references
  1. [9]

    Unsuper- vised scale-consistent depth and ego-motion learning from monocular video

    Jiawang Bian, Zhichao Li, Naiyan Wang, Huangying Zhan, Chunhua Shen, Ming-Ming Cheng, and Ian Reid. Unsuper- vised scale-consistent depth and ego-motion learning from monocular video. Advances in Neural Information Process- ing Systems, 32, 2019. 1

    2019

  2. [10]

    The platonic representation hypothesis

    Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The platonic representation hypothesis. arXiv preprint arXiv:2405.07987, 2024. 2, 3

    2024 arXiv

  3. [11]

    Occlusion aware unsupervised learning of optical flow

    Yang Wang, Yezhou Yang, Zhenheng Yang, Liang Zhao, and Wei Xu. Occlusion aware unsupervised learning of optical flow. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4884–4893, 2017. 2, 3

    2018

  4. [12]

    Un- supervised learning of depth and ego-motion from monocu- lar video using 3d geometric constraints

    Reza Mahjourian, Martin Wicke, and Anelia Angelova. Un- supervised learning of depth and ego-motion from monocu- lar video using 3d geometric constraints. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5667–5675, 2018. 2, 3

    2018

  5. [13]

    Controlnext: Powerful and effi- cient control for image and video generation

    Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming- Chang Yang, and Jiaya Jia. Controlnext: Powerful and effi- cient control for image and video generation. arXiv preprint arXiv:2408.06070, 2024. 2, 4

    2024 arXiv

  6. [14]

    Self- supervised monocular depth and ego-motion estimation in endoscopy: Appearance flow to the rescue

    Shuwei Shao, Zhongcai Pei, Weihai Chen, Wentao Zhu, Xingming Wu, Dianmin Sun, and Baochang Zhang. Self- supervised monocular depth and ego-motion estimation in endoscopy: Appearance flow to the rescue. Medical Image Analysis, 77:102338, 2022. 2, 3, 4, 5, 13

    2022

  7. [15]

    Diffusion mod- els and representation learning: A survey

    Michael Fuest, Pingchuan Ma, Ming Gui, Johannes S Fis- cher, Vincent Tao Hu, and Bjorn Ommer. Diffusion mod- els and representation learning: A survey. arXiv preprint arXiv:2407.00783, 2024. 3

    2024 arXiv

  8. [16]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022. 3, 5

    2022

  9. [17]

    Maximum likelihood training of implicit nonlinear diffusion model

    Dongjun Kim, Byeonghu Na, Se Jung Kwon, Dongsoo Lee, Wanmo Kang, and Il-Chul Moon. Maximum likelihood training of implicit nonlinear diffusion model. Advances in Neural Information Processing Systems , 35:32270–32284,

  10. [18]

    Your gan is secretly an energy-based model and you should use discriminator driven latent sampling

    Tong Che, Ruixiang Zhang, Jascha Sohl-Dickstein, Hugo Larochelle, Liam Paull, Yuan Cao, and Yoshua Bengio. Your gan is secretly an energy-based model and you should use discriminator driven latent sampling. Advances in Neural Information Processing Systems, 33:12275–12287, 2020. 3

    2020

  11. [19]

    On tracking the partition function

    Guillaume Desjardins, Yoshua Bengio, and Aaron C Courville. On tracking the partition function. Advances in Neural Information Processing Systems, 24, 2011. 3

    2011

  12. [20]

    Implicit generation and mod- eling with energy based models

    Yilun Du and Igor Mordatch. Implicit generation and mod- eling with energy based models. Advances in Neural Infor- mation Processing Systems, 32, 2019. 3

    2019

  13. [21]

    Your vit is secretly a hybrid discriminative-generative diffusion model

    Xiulong Yang, Sheng-Min Shih, Yinlin Fu, Xiaoting Zhao, and Shihao Ji. Your vit is secretly a hybrid discriminative-generative diffusion model. arXiv preprint arXiv:2208.07791, 2022. 3

    2022 arXiv

  14. [23]

    Denoising diffusion autoencoders are unified self-supervised learners

    Weilai Xiang, Hongyu Yang, Di Huang, and Yunhong Wang. Denoising diffusion autoencoders are unified self-supervised learners. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15802–15812, 2023. 3

    2023

  15. [24]

    Dreamteacher: Pretraining image backbones with deep generative models

    Daiqing Li, Huan Ling, Amlan Kar, David Acuna, Se- ung Wook Kim, Karsten Kreis, Antonio Torralba, and Sanja Fidler. Dreamteacher: Pretraining image backbones with deep generative models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16698– 1...

    2023

  16. [26]

    Diffusion models beat gans on image classification

    Soumik Mukhopadhyay, Matthew Gwilliam, Vatsal Agar- wal, Namitha Padmanabhan, Archana Swaminathan, Srinidhi Hegde, Tianyi Zhou, and Abhinav Shrivastava. Diffusion models beat gans on image classification. arXiv preprint arXiv:2307.08702, 2023. 3, 12

    2023 arXiv

  17. [27]

    Learning data representations with joint diffusion mod- els

    Kamil Deja, Tomasz Trzci ´nski, and Jakub M Tomczak. Learning data representations with joint diffusion mod- els. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases , pages 543–559. Springer, 2023. 3, 12

    2023

  18. [28]

    Dreamteacher: Pretraining image backbones with deep generative models

    Daiqing Li, Huan Ling, Amlan Kar, David Acuna, Se- ung Wook Kim, Karsten Kreis, Antonio Torralba, and Sanja Fidler. Dreamteacher: Pretraining image backbones with deep generative models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16698– 1...

    2023

  19. [29]

    Unleashing text-to-image diffu- sion models for visual perception

    Wenliang Zhao, Yongming Rao, Zuyan Liu, Benlin Liu, Jie Zhou, and Jiwen Lu. Unleashing text-to-image diffu- sion models for visual perception. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 5729–5739, 2023. 3, 12

    2023

  20. [30]

    Diffusion model as repre- sentation learner

    Xingyi Yang and Xinchao Wang. Diffusion model as repre- sentation learner. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 18938–18949,

  21. [31]

    Con- trastive multiview coding

    Yonglong Tian, Dilip Krishnan, and Phillip Isola. Con- trastive multiview coding. In European Conference on Com- puter Vision, 2020. 3

    2020

  22. [32]

    Contrastive learning inverts the data generating process

    Roland S Zimmermann, Yash Sharma, Steffen Schneider, Matthias Bethge, and Wieland Brendel. Contrastive learning inverts the data generating process. In International Con- ference on Machine Learning, pages 12979–12990. PMLR,

  23. [33]

    Brown, Noah Snavely, and David G

    Tinghui Zhou, Matthew A. Brown, Noah Snavely, and David G. Lowe. Unsupervised learning of depth and ego- motion from video. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 6612–6619,

    2017

  24. [34]

    Self-supervised learning with geometric constraints in monocular video: Connecting flow, depth, and camera

    Yuhua Chen, Cordelia Schmid, and Cristian Sminchis- escu. Self-supervised learning with geometric constraints in monocular video: Connecting flow, depth, and camera. 2019 IEEE/CVF International Conference on Computer Vi- sion (ICCV), pages 7062–7071, 2019. 3

    2019

  25. [35]

    Jiawang Bian, Zhichao Li, Naiyan Wang, Huangying Zhan, Chunhua Shen, Ming-Ming Cheng, and Ian D. Reid. Un- supervised scale-consistent depth and ego-motion learning from monocular video. In Neural Information Processing Systems, 2019. 3

    2019

  26. [36]

    Carneiro

    Adrian Johnston and G. Carneiro. Self-supervised monocu- lar trained depth estimation using self-attention and discrete disparity volume. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4755–4764,

    2020

  27. [37]

    Bros- tow

    Cl ´ement Godard, Oisin Mac Aodha, and Gabriel J. Bros- tow. Digging into self-supervised monocular depth estima- tion. 2019 IEEE/CVF International Conference on Com- puter Vision (ICCV), pages 3827–3837, 2018. 3, 5, 6

    2019

  28. [38]

    Self-supervised lightweight depth estimation in endoscopy combining cnn and transformer

    Zhuoyue Yang, Junjun Pan, Ju Dai, Zhen Sun, and Yi Xiao. Self-supervised lightweight depth estimation in endoscopy combining cnn and transformer. IEEE Transactions on Med- ical Imaging, 43:1934–1944, 2024. 3, 5

    1934

  29. [39]

    Shuwei Shao, Zhongcai Pei, Weihai Chen, Dingchi Sun, Peter C. Y . Chen, and Zhengguo Li. Monodiffusion: Self-supervised monocular depth estimation using diffusion model. arXiv preprint arXiv:2311.07198, 2023. 3, 5, 6

    2023 arXiv

  30. [40]

    Generative ad- versarial text to image synthesis

    Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Lo- geswaran, Bernt Schiele, and Honglak Lee. Generative ad- versarial text to image synthesis. InInternational Conference on Machine Learning, pages 1060–1069. PMLR, 2016. 4

    2016

  31. [41]

    Image-to-image translation with conditional adver- sarial networks

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adver- sarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 1125– 1134, 2017. 4

    2017

  32. [42]

    Semantic image synthesis with spatially-adaptive nor- malization

    Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu. Semantic image synthesis with spatially-adaptive nor- malization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2337– 2346, 2019. 4

    2019

  33. [43]

    3d reconstruction from endoscopy images: A survey

    Zhuoyue Yang, Ju Dai, and Junjun Pan. 3d reconstruction from endoscopy images: A survey. Computers in Biology and Medicine, page 108546, 2024. 4

    2024

  34. [44]

    Spatial transformer networks

    Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. Spatial transformer networks. Advances in Neural Informa- tion Processing Systems, 28, 2015. 4

    2015

  35. [45]

    Stereo correspondence and recon- struction of endoscopic data challenge

    Max Allan, Jonathan Mcleod, Congcong Wang, and Jean- Claude Rosenthal. Stereo correspondence and recon- struction of endoscopic data challenge. arXiv preprint arXiv:2101.01133, 2021. 5

    2021 arXiv

  36. [46]

    Durr, Hunter B

    Kutsev Bengisu Ozyoruk, Guliz Irem Gokceler, Gulfize Coskun, Kagan Incetan, Yasin Almalioglu, Faisal Mahmood, Eva Curto, Luis Perdigoto, Marina Oliveira, Hasan Sahin, Helder Ara ´ujo, Henrique Alexandrino, N. Durr, Hunter B. Gilbert, and Mehmet Turan. Endoslam dataset and an u...

    2021

  37. [47]

    Automatic differentiation in pytorch

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Al- ban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017. 5

    2017

  38. [48]

    Masked diffu- sion as self-supervised representation learner

    Zixuan Pan, Jianxu Chen, and Yiyu Shi. Masked diffu- sion as self-supervised representation learner. arXiv preprint arXiv:2308.05695, 2023. 12

    2023 arXiv

  39. [49]

    Addp: Learning general representations for image recognition and generation with alternating denoising diffusion process

    Changyao Tian, Chenxin Tao, Jifeng Dai, Hao Li, Ziheng Li, Lewei Lu, Xiaogang Wang, Hongsheng Li, Gao Huang, and Xizhou Zhu. Addp: Learning general representations for image recognition and generation with alternating denoising diffusion process. arXiv preprint arXiv:2306.0542...

    2023 arXiv

  40. [50]

    Robust agents learn causal world models

    Jonathan Richens and Tom Everitt. Robust agents learn causal world models. arXiv preprint arXiv:2402.10877 ,

    arXiv

  41. [51]

    A vision check-up for language models

    Pratyusha Sharma, Tamar Rott Shaham, Manel Baradad, Stephanie Fu, Adrian Rodriguez-Munoz, Shivam Duggal, Phillip Isola, and Antonio Torralba. A vision check-up for language models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 14...

    2024

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.